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Preface to the second volume of Model Order 
Reduction 


This second volume of the Model Order Reduction handbook project mostly focuses 
on snapshot-based methods for parameterized partial differential equations. This ap- 
proach has seen tremendous development in the past two decades, especially in the 
broad domain of computational mechanics. However, the main ideas were already 
known long before; see, e. g., the seminal work by J. L. Lumley, “The structure of in- 
homogeneous turbulent flows,” in Atmospheric Turbulence and Radio Wave Propaga- 
tion, 1967, for proper orthogonal decomposition (POD), and the one by A. K. Noor and 
J. N. Peters, Reduced basis technique for nonlinear analysis of structures, AIAA Journal, 
Vol. 4, 1980, for the reduced basis method. 

The most popular mathematical strategy behind snapshot-based methods relies 
on Galerkin projection on finite-dimensional subspaces generated by snapshot so- 
lutions corresponding to a special choice of parameters. Because of that, it is often 
termed as a projection-based intrusive approach. A suitable offline-online splitting of 
the computational steps, as well as the use of hyperreduction techniques to be used 
for the nonlinear (or nonaffine) terms and nonlinear residuals, is key to efficiency. 

The first chapter, by G. Rozza etal., introduces all the preliminary notions and 
basic ideas to start delving into the topic of snapshot-based model order reduction. 
All the notions will be recast into a deeper perspective in the following chapters. 

The second chapter, by Grassle et al., provides an introduction to POD with a focus 
on (nonlinear) parametric partial differential equations (PDEs) and (nonlinear) time- 
dependent PDEs, and PDE-constrained optimization with POD surrogate models as 
application. Several numerical examples are provided to support the theoretical find- 
ings. 

A second scenario in the methodological development is provided in the third 
chapter, by Chinesta and Ladevéze, on proper generalized decomposition, a research 
line significantly grown in the last couple of decades also thanks to real-world applica- 
tions. Basic concepts used here rely on the separation of variables (time, space, design 
parameters) and tensorization. 

The fourth chapter, by Maday and Patera, focuses on the reduced basis method, 
including a posteriori error estimation, as well as a primal-dual approach. Several 
combinations of these approaches have been proposed in the last few years to face 
problems of increasing complexity. 

When facing nonaffine and nonlinear problems, the development of efficient re- 
duction strategies is of paramount importance. These strategies can require either 
global or local (pointwise) subspace constructions. This issue is thoroughly covered in 
the fifth chapter, by Farhat et al., where several front-end computational problems in 
the field of nonlinear structural dynamics, scattering elastoacoustic wave propagation 
problems, and a parametric PDE-ODE wildfire model problem are presented. 
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In the sixth chapter, by Buhr et al., localized model order reduction is presented. 
With this approach, the model order reduction solution is constructed via a suitable 
coupling of local solutions whose support lies within a subdomain of the global com- 
putational domain. Applications are provided for multiscale, linear elasticity, and 
fluid-flow problems. 

Last but not least, the final chapter, by Brunton and Kutz, addresses a snapshot- 
based nonintrusive data-driven method. In particular, dynamic mode decomposition 
and its Koopman generalization are used to discover low-rank spatio-temporal pat- 
terns of activity, and to provide approximations in terms of linear dynamical systems, 
which are amenable to simple analysis techniques. These methods can be used in a 
nonintrusive, equation-free manner for improved computational performance of para- 
metric PDE systems. 

Several chapters contain instructive descriptions of algorithms that can serve as 
templates for implementing the discussed approaches in problem-specific environ- 
ments. 
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1 Basic ideas and tools for projection-based 
model reduction of parametric partial 
differential equations 


Abstract: We provide first the functional analysis background required for reduced- 
order modeling and present the underlying concepts of reduced basis model reduc- 
tion. The projection-based model reduction framework under affinity assumptions, 
offline-online decomposition, and error estimation are introduced. Several tools for 
geometry parameterizations such as free form deformation, radial basis function in- 
terpolation, and inverse distance weighting interpolation are explained. The empirical 
interpolation method is introduced as a general tool to deal with nonaffine parameter 
dependency and nonlinear problems. The discrete and matrix versions of the empir- 
ical interpolation are considered as well. Active subspace properties are discussed to 
reduce high-dimensional parameter spaces as a preprocessing step. Several examples 
illustrate the methodologies. 


Keywords: reduced basis method, radial basis function interpolation, shape morphing 
techniques, empirical interpolation method, active subspaces 


MSC 2010: 65D99, 65J05, 65M15 


Introduction 


Parametric model order reduction (MOR) techniques have been developed in recent 
decades to deal with increasingly complex computational tasks. The ability to com- 
pute how quantities of interest change with respect to parameter variations provides 
insight and understanding, which is vital in all areas of science and engineering. 
Model reduction thus allows to deal with optimization or inverse problems of a whole 
new scale. Each chapter of the handbook gives an in-depth view of a MOR method, 
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a particular application area, and analytical, numerical, or technical aspects of soft- 
ware frameworks for model reduction. 

There exist a large number of MOR techniques used in many areas of science and 
engineering to improve computational performances and contain costs in a repeti- 
tive computational environment, such as many-query and real-time computing [93]. 
We assume a given parameterized partial differential equation (PDE) as starting point 
of the model reduction procedure. Typical parameters of interest are material coeffi- 
cients, corresponding to physical qualities of the media which constitute the domain 
where the PDE is solved. Also a variable geometry can be of special interest in a task to 
find the optimal device configuration. Physical states such as the temperature might 
be considered an input parameter. It is a task of the mathematical modeling to identify 
the parameters of interest and how they enter the PDE. Once a parameterized model 
is identified, the MOR techniques described in this and the following chapters can be 
used either in a “black-box” fashion (nonintrusive way) or by intrusive means, which 
will be explained in detail, whenever this is necessary. 

The particular numerical method to solve a PDE is most often not relevant to the 
model reduction procedure. We will therefore assume there is a numerical method 
available, which solves the problem to any required accuracy, and move seamlessly 
from the continuous form to the discretized form. 

This chapter covers briefly the functional analysis framework relevant to many, 
but not all, MOR methods. Presented is the starting point of PDE-oriented MOR tech- 
niques, such as the POD method found in Chapter 2 of this volume, the PGD method 
found in Chapter 3 of this volume, the reduced basis method found in Chapter 4 of this 
volume, the hyperreduction technique found in Chapter 5 of this volume, the localized 
reduced-order modeling (ROM) found in Chapter 6 of this volume, and the data-driven 
methods found in Chapter 7 of this volume. 

In particular, Section 1.1 provides what is needed for the projection-based ROM. 
Starting from the setting of the classical Lax—-Milgram theorem for elliptic PDEs in Sec- 
tions 1.1.1 and 1.1.2, a numerical discretization is introduced in Section 1.1.2.1. Due to 
brevity of representation, many concepts of functional analysis and theory of PDEs 
are only touched upon. Many references to the literature for further reading are given. 

Projection-based ROM is presented in Section 1.1.3, with the following topics cov- 
ered in detail: proper orthogonal decomposition (POD) in Section 1.1.3.1, the greedy 
algorithm in Section 1.1.3.2, the projection framework in Section 1.1.3.3, affine param- 
eter dependency in Section 1.1.3.4, the offline-online decomposition in Section 1.1.3.6, 
and basic error estimation in Section 1.1.4. 

Section 1.2 introduces efficient techniques for geometric parameterizations, aris- 
ing from a reference domain approach, such as free form deformation (FFD) in Sec- 
tion 1.2.1, radial basis function (RBF) interpolation in Section 1.2.2, and inverse dis- 
tance weighting (IDW) in Section 1.2.3. 

A widely used method to generate an approximate affine parameter dependency is 
the empirical interpolation method (EIM). The original EIM is presented in Section 1.3 
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as well as the discrete EIM in Section 1.3.3 and further options in Section 1.3.4. Several 
numerical examples show the use of the EIM in Section 1.3.5. 

Section 1.4 introduces active subspaces as a preprocessing step to reduce the pa- 
rameter space dimension. Corresponding examples are provided in Section 1.4.3 and 
also nonlinear dimensionality reduction is briefly discussed in Section 1.4.5. 

A brief conclusion and an outlook of the handbook are given in Section 1.5. 


1.1 Basic notions and tools 


We briefly cover a few main results of linear functional analysis and the analysis of 
PDEs. This material serves as a reminder of the underlying concepts of model reduc- 
tion but cannot replace a textbook on these subjects. For amore thorough background, 
we refer to the literature on functional analysis [30, 110], PDEs [1, 47, 82, 88], and nu- 
merical methods [2, 6, 29, 52, 80, 105]. 


1.1.1 Parameterized partial differential equations 


Let Q c IR? denote a spatial domain in d = 1,2, or 3 dimensions with boundary 0Q. 
A Dirichlet boundary Ip c 0Q is given, where essential boundary conditions on the 
field of interest are prescribed. Introduce a Hilbert space V(Q) equipped with inner 
product (.,-), and induced norm ||- |y. A Hilbert space V(Q) is a function space, i. e., a 
function u € V(Q) is seen as a point in the vector space V, as is common in functional 
analysis. Each u € V(Q) defines a mapping x € OQ» u(x) € Rorx € Q > u(x) € C, 
depending on whether a real or complex Hilbert space is considered. In many appli- 
cations, V is a subset of the Sobolev space H'(Q) as V(Q) = {v € H (O): vlr, = 0}. 
Vector-valued Hilbert spaces can be constructed using the Cartesian product of V (Q). 
Given a parameter domain P c R”, a particular parameter point is denoted by the 
p-tuple u = (Hy, Mp, . . -, Hp). The set of all linear and continuous forms on V defines the 
dual space V’; let L € L(V, V’) denote a linear differential operator. 

A field variable u € V : Q — Ris defined implicitly as the solution to a parameter- 
ized linear PDE through the operator L : V x P > V' with L(;p) € £(V, V’) and load 
vector f; (u) € V’ for each fixed y, as 


L(u; p) = f(y). (1.1) 


As in the case of function spaces, operators between function spaces form vector 
spaces themselves, such as L(; u) € L(V, V’), with L(V, V’) being the space of opera- 
tors mapping from the vector space V to V’. 

Typical examples of scalar-valued linear PDEs are the Poisson equation, the heat 
equation, and the wave equation, while typical examples of vector-valued linear PDEs 
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are the Maxwell equations and the Stokes equations. The nonlinear case will be ad- 
dressed in various chapters as well: Examples of nonlinear PDEs include the Navier- 
Stokes system and the equations describing nonlinear elasticity. 


1.1.2 Parameterized variational formulation 


The variational form or weak form of a parameterized linear PDE in the continuous 
setting is given as 


a(u(H),v;H) =f(v;p) Wwe, (1.2) 


with bilinear form a : V x V x P — R and linear form f : V xP — R. In many 
application scenarios, a particular output of interest is sought, given by the linear form 
1:VxP —>Ras 


s(p) = I(u(p); p). (1.3) 


In the case that a(., -; u) is symmetric and l = f, the problem is called compliant. 
For each u € P assume coercivity and continuity of the bilinear form a(., -; y), i. e., 


a(w, ws p) = a(p)liwlly,, (1.4) 
a(w, v; y) sy) |lwilyllvly, (1.5) 


and continuity of the linear form f(-; y), 


fw; u) < 6(wllwlly, (1.6) 


with parameter-independent bounds, which satisfy O < a < a(u), y(u) < y < co, and 
ô(u) < 6 < oo. To do actual computations, the bilinear form is discretized into a linear 
equation. The coercivity property means that the matrix discretizing the bilinear form 
will be positive definite. 

For fixed parameter the well-posedness of (1.2) is then established by the Lax- 
Milgram theorem. 


Theorem 1.1 (Lax—Milgram theorem). Leta: V x V — R be a continuous and coercive 
bilinear form over a Hilbert space V and f € V' a continuous linear form. Then the 
variational problem 


a(u,v)=f(v) Wwev (1.7) 
has a unique solution u € V and we have 
1 
luly < zl (1.8) 


with the coercivity constant a > 0 of the bilinear form. 
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Thus, in the parametric setting, the -dependence also carries over to the coerciv- 
ity constant as a = a(p). 

The function space in which the field variable resides is called the ansatz space, 
while the second function space is called the test space, i. e., where a test function v 
resides. If the test space is distinct from the ansatz space, then the bilinear form is 
defined over a : Vx Wx P — R for V and W Hilbert spaces. With f € W’ and for 
fixed u, the well-posedness is then established through the Banach-Necas—BabuSka 
theorem. 


Theorem 1.2 (Banach—Neéas—BabuSka theorem). Let V and W denote Hilbert spaces, 
leta : Vx W > R be a continuous bilinear form, and f € W'. Then the variational 
problem 


a(u,v) =f(v) Wwe W (1.9) 
has a unique solution if and only if 
(i) the inf-sup condition holds, i. e., 


WB>0,s.t,B< inf aW- 
veV\10} wew\;o} lvliyliwlw 


(ii) Yw €e W: 


{a(v,w) =OVve V} => w=0. 


1.1.2.1 Discretized parameterized variational formulation 


The method of weighted residuals is used to cast (1.1) into a discrete variational 
formulation. Given the linear PDE L(u;u) = f(y), consider a discrete, i.e., finite- 
dimensional, approximation u, € V} c V to u as 


Up, (H) = 2, ugi. (1.10) 


The dimension of V, is N, and the set of ansatz functions `x) : Q —> R belong 
to V. The u® are scalar coefficients such that the vector u, = Cu”, siss Ta € Ris 
the coordinate representation of u, in the basis {@'} of V,. A conforming discretization 
is considered, i. e., V, c V holds. 

Plugging (1.10) into (1.1) yields the discrete residual R(u,(4)) = L(u W); p) - 
fı) < V'. To compute the scalar coefficients u®, Galerkin orthogonality is invoked, 
as 


0= (p; R), v) j si, .. Np; (1.11) 


where (-, -)(y,y) is the duality pairing between V and V'. 
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In short, Galerkin orthogonality means that the test space is orthogonal to the 
residual. In Ritz—Galerkin methods, the residual is tested against the same set of func- 
tions as the ansatz functions. If test space and trial space are different, one speaks of 
a Petrov-Galerkin method. Numerous discretization methods can be understood in 
terms of the method of weighted residuals. They are distinguished by the particular 
choice of trial and test space. 

The well-posedness of the discrete setting follows the presentation of the contin- 
uous setting, by casting the equations and properties over V, instead of V. 

The weak form in the discrete setting is given as 


a(up(H), Vins) =F (vps) Wh € Vis (1.12) 


with bilinear form a : Vp x Vp x P — Rand linear form f : Vp x P > R. The discrete 
bilinear form is then derived from (1.11) through the integration-by-parts formula and 
Green’s theorem. 

Correspondingly, the discrete coercivity constant a, (u) and the discrete continuity 
constant y,(y) are defined as 


a, (HW) = min AWe Wip) (1.13) 
VA wall, 
a(Wh Vn; H) 


Yn(H) = max max (1.14) 


WhEVh VnE Va Wally, IIVallv, 

The well-posedness of (1.2) is then analogously established by the Lax—Milgram 

theorem and the Banach—Ne¢as—Babu&ka theorem. Cea’s lemma is a fundamental re- 
sult about the approximation quality that can be achieved. 


Lemma 1.3 (Cea’s lemma). Leta : V x V — R bea continuous and coercive bilinear 
form over a Hilbert space V and f < V' a continuous linear form. Given a conforming 
finite-dimensional subspace V, c V, the continuity constant y, and coercivity constant 
a of a(-,-), for the solution up to 


alUn Vn) =f Vh) Yp € Vp (1.15) 
we have 


z Y i a 
lu —Uplly < eo lu — Vally- (1.16) 


The stiffness matrix A} € R~” assembles the bilinear form entrywise as (Ani = 
alo gy). The load vector f, € R^? is assembled entrywise as (f,); = f (g') and the 
solution vector is denoted u, with coefficients u®, 

Then solving (1.12) amounts to solving the linear system 


Ahun = f,. (1.17) 
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The most common discretization method is the finite element method [13], besides 
the finite difference [97], discontinuous Galerkin [4], finite volume [48], and spectral 
element methods [19]. 


1.1.3 Model reduction basic concepts 


A wide variety of ROM methods exist today, thanks to large research efforts in the 
last decades. Reduced basis MOR is a projection-based MOR method and also shares 
many features with other MOR methods, so that the topics mentioned here will occur 
throughout the handbook. Two common algorithms for the generation of a projection 
space, POD and the greedy algorithm, are presented first. 


1.1.3.1 Proper orthogonal decomposition 


Assume a sampled set of high-fidelity solutions {u),(u;),i = 1,...,Nmax}, i. €., solutions 
to (1.12) or (1.17), respectively. The discrete solution vectors are stored columnwise in a 
snapshot matrix $ € IRN**Nmax, POD compresses the data stored in $ by computing an 
orthogonal matrix V, which is a best approximation in the least-squares sense to S. In 
particular, the POD solution of size N is the solution to 


min _||S—Vv’S\|,, (1.18) 
VeRnnx’ 
subject to VV = Iyxy> (1.19) 


with || - ||; being the Frobenius norm and Iyxy being the identity matrix. 
There exists a solution to (1.18)—(1.19) according to the Eckardt-Young—Mirsky the- 
orem [43], which can be computed with singular value decomposition (SVD) as 


S = UXZ, (1.20) 


with orthogonal matrix U € RM, rectangular diagonal matrix = € RMN, and 
orthogonal matrix Z € RNra*Nmax, The solution V is composed of the first N col- 
umn vectors of U. They are also called the POD modes. The diagonal entries {0;,i = 
1,...,min(N), Nmax)} of È are nonnegative and are called singular values. We have 


min(Np,Nmax) 
: T 
min |S-VV S|, = Oj. (1.21) 
VeR nN lr a. ' 


Thus, the neglected singular values give an indication of the approximate trunca- 
tion error. In practise, a high tolerance threshold like 99 % or 99.99 % is chosen and 
N is determined so that the sum of the first N singular values reaches this percentage 
of the sum of all singular values. In many applications, an exponential singular value 
decay can be observed, which allows to reach the tolerance with a few POD modes. 
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1.1.3.2 Greedy algorithm 


The greedy algorithm also computes an orthogonal matrix V € RY to serve as a 
projection operator, just as in the POD case. The greedy algorithm is an iterative pro- 
cedure, which enriches the snapshot space according to where an error indicator or 
error estimator A attains its maximum. Starting from a field solution at a given initial 
parameter value, the parameter location is sought, whose field solution is worst ap- 
proximated with the initial solution. This solution is then computed and appended to 
the projection matrix to obtain a two-dimensional projection space. The greedy typi- 
cally searches for new snapshot solutions within a discrete surrogate P of the parame- 
ter space P. The process is repeated until a given tolerance on the error estimator is ful- 
filled. The error estimator is residual-based and estimates the error between a reduced- 
order solve for a projection space V and the high-fidelity solution (Section 1.1.4). The 
greedy algorithm is stated in pseudo-code in Algorithm 1.1. 


Algorithm 1.1: The greedy algorithm. 

Input: discrete surrogate P of parameter space P, approximation tolerance tol, 
initial parameter 4, 

Output: projection matrix V 
N= 


while max,<p A(H) > tol do 
N=N+1 


Hy = arg max A(p) 
peP 


solve (1.17) at uy for up (Hy) 
orthonormalize u, (My) with respect to Yy- to obtain ¢y 
append ¢y to Vy- to obtain Vy 

end while 

set V = Vy 


1.1.3.3 Reduced-order system 


Starting from the discrete high-fidelity formulation (1.12), another Galerkin projection 
is invoked to arrive at the reduced-order formulation. Assume a projection space Vy 
is then determined through either a POD or the greedy sampling, with V € RN" 
denoting a discrete basis of Vy. Thus Vy c Vp and dim Vy = N. 

The reduced-order variational formulation is to determine uy (H) € Vy, such that 


a(uy (BK), Vy) = f(v) Wy € Vy. (1.22) 
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Equation (1.17) is then projected onto the reduced-order space as 
V'A,Vuy = V'f,. (1.23) 


The reduced system matrix Ay = V’A,,V is then a dense matrix of small size N xN 
as depicted in (1.24): 


alg’, g’) 
[An]= |Y oe ote Vi. (1.24) 
alo™ z gh) 
The high-order solution is then approximated as 
u, = Vuy. (1.25) 


1.1.3.4 Affine parameter dependency 


Many MOR algorithms rely on an affine parameter dependency, because the affine 
parameter dependency provides the computational efħciency of the model reduc- 
tion. Thus, it is a significant advancement from the 2000s [85] over the first use of 
ROMs [3, 76]. 

An affine parameter dependency means that the bilinear form can be expanded 


as 
Qa 
a(l, sp) = > OL (wa;(-,-), (1.26) 
i=1 
and affine expansions hold as 
Qr 
fom => OAC, (1.27) 
i=1 
Q 
Isp) = È OWO, (1.28) 


i=1 


with scalar-valued functions oi :P>R, o; : P > R, and o :P>R. 
Correspondingly the linear system (1.17) can be expanded as 


Q G 
(> OLDA; Jen =) OW f;, (1.29) 
i=1 i=1 


as well as the reduced-order form (1.23) 


Qao 6 Qe 
0) Olas Wy = V” È OQ, (1.30) 


i=1 i=1 
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Qa Qr 
(> oL VAN Jux DA (1.31) 
i=1 i=1 

MOR relies on an affine parameter dependency, such that all computations de- 
pending on the high-order model size can be moved into a parameter-independent 
offline phase, while having a fast input-output evaluation online. If the problem is 
not affine, an affine representation can be approximated using a technique such as 
the EIM (Section 1.3). 


1.1.3.5 Affine shape parameterizations: an example 


Consider heat conduction in a square domain Q(x, y) = [0,1]*. On the left side x = 0, 
inhomogeneous Neumann conditions, i. e., a nonzero heat flux, are imposed and on 
the right side x = 1, homogeneous Dirichlet conditions, i.e., zero temperature, are 
imposed. On the top and bottom sides, homogeneous Neumann conditions, i. e., a zero 
heat flux, are imposed. Consider two different media with different conductivities o, 
and o, occupying the subdomains Q, (4) = [0, u] x [0,1] and 0,(w) = [p, 1] x [0, 1], for 
u € P = (0,1), as shown in Figure 1.1. For the sake of clarity, in the rest of this section 
we identify the one-dimensional parameter vector u with its (only) component py, thus 
dropping the bold notation from the symbol. 


(0,1) aa) 
Q, Q, 
y : 
z ; Figure 1.1: The computational domain is subdivided into 
: two domains Q = Q, U Q,, depending on the parameter p. 
(0,0) (H,0) (1,0) Shown here for p = 0.5. 


Choosing 4 = 0.5 as the reference configuration, there exist affine transformations 
from the reference domain to the actual domain. We have 


Ti : QH) > QH) : OGY) => 2x, y), (1.32) 
T : OF) > QH) : ®Y) > ((2- 2u), F) + (2u - 1, 0). (1.33) 


In general, an affine transformation of a subdomain can be expressed as 
Ty : Q) > Qu) : x G,(u)x + Dg), (1.34) 


with x € RÎ, G, € R®@ and D, € Rf in d = 2,3 spatial dimensions. 
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Thus, the bilinear form 


a(u, V; u) = | 0,Vu- Vvdx + | 07Vu- Vv dx (1.35) 
Q) Q24) 


can be mapped to the reference domain with the inverse affine transformation 
TZ’ : O(n) > OM : x G(x - GWD), (1.36) 


and integration by substitution as 


a(u, v; u) = | 0,(VuG;'(u)) - (GI Qu) Vv) det(G,(u)) dx (1.37) 
Q;(H) 
+ | 0(VuG31(u)) « (G5" Quvv) det(Go(u)) dx, (1.38) 
op) 


which establishes the affine parameter dependency (1.26) by computing oi (u) from 
the coefficients of G, and G, [85, 84, 28]. That is, 


| 0, (VuG;1()) - (G77 QVv) det(G,(0)) dx (1.39) 
am 
= | 0((2u) ‘du, Oyu) - ((2u) ‘dv, Oyv)2u dx (1.40) 
oD 
= (20) | 0; (2,1) (0,v) dX + 2u | 0,(0,u)(0,v) dx, (1.41) 
am a 
and 
| 0(VuG3!(u)) « (GZT QV) det(G3(u)) dx (1.42) 
om) 
= | 0((2 — 24) 'd,u, du) - ((2 - 2) dv, 8,v)(2 - 2p) dx (1.43) 
OM) 
=(2- 2u)* | 0(0,u)(0,Vv) dX + (2 — 24) | 0(d,u)(d,V) dx, (1.44) 
on o 


which establishes the affine form (1.26) with Q, = 4, and 


olu) = 29W", (1.45) 
@? (u) = 2p, (1.46) 
Ožu) = (2 - 2p)", (1.47) 
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@4(u) =2- 2p, (1.48) 
and 

al :) = | 0,(0,u)(d,Vv) dX, (1.49) 

a 
a,(-,-) = | 0; (yu) (Ayv) dx, (1.50) 

am 
a3(-,-) = | 07(0,u)(0,Vv) ax, (1.51) 

2, 
Oe | 03(0,u)(2yv) dx. (1.52) 

o(p) 


The second and fourth terms can be further simplified to a term depending on 
2u and a -independent term, but in this case it still leaves Q, = 4 terms. In some 
cases the number of affine terms can be automatically reduced further using symbolic 
computations. 


1.1.3.6 Offline-online decomposition 


The offline-online decomposition enables the computational speedup of the ROM 
approach in many-query scenarios. It is also known as the offline-online paradigm, 
which assumes that a computation-intensive offline phase can be performed on a 
supercomputer, which generates all quantities depending on the large discretization 
size N}. Once completed, a reduced-order solve, i.e., an online solve for a new pa- 
rameter of interest, can be performed with computational cost independent of the 
large discretization size N,,. The online phase can thus be performed even on mobile 
and embedded devices (Figure 1.2). If a supercomputer is not available, this can be 
relaxed, however. There exist heuristic algorithms to make also the offline phase fea- 
sible on a common workstation, such that a typical scenario would be that the offline 
phase runs overnight and a reduced model is available the next morning. 


OFFLINE PHASE " ONLINE PHASE 
Real time 
A g = £—*) simulations E 
orog [renra | —— 
ere} | ameo al 
oura ummo — =v 
HPC cluster Portable devices 


Figure 1.2: Offline-online paradigm. The complex high-fidelity simulations are carried out in high 
performance clusters (HPCs) for given preselected parameters. The solution snapshots can be stored 
and the ROM trained. Then in the offline phase the ROM provides approximated solutions at new 
untried parameters in real-time on simple portable devices. 
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Noting that the terms V’A,V and V’f, in (1.31) are parameter-independent, they can 
be precomputed, prior to any ROM parameter sweep. This will store small-sized dense 
matrices of dimension N x N. Once a reduced-order solution uy is desired for a given 
parameter u, the sum given in (1.31) is formed and solved for uy. Since this is the same 
as solving (1.23), the reduced-order approximation is then available as u, = Vuy; 
see (1.25). 


1.1.4 Error bounds 


In this section we develop effective and reliable a posteriori error estimators for the 
field variable or an output of interest. The use of such error bounds drives the con- 
struction of the reduced basis during the offline stage, thanks to the so-called greedy 
algorithm. Moreover, during the online stage, such bounds provide a certified accu- 
racy of the proposed ROM. 

Following [85], we introduce residual-based a posteriori error estimation for the 
elliptic case. From (1.12) and (1.22) it follows that the error e(u) = up()- uy (H) satisfies 


a(e(H), Vw H) = "(Vas H) Wh € Vp; (1.53) 
where the residual r(.; u) € V} is defined as 
T(V W) =f (Vp p) -aluy(H), VH) Yh € Vp- (1.54) 


The following theorem further characterizes the relation between error and resid- 
ual: 


Theorem 1.4. Under compliance assumptions, the following inequalities hold: 


Irswlly; 
[edy = Neen) - Uy) Ip < Aen (H) = Rar (1.55) 
h 
Ir WI? 
O < Sp(H) - sn (H) < AsH) = S (1.56) 


where |iv|?}, = a(v, v; u) defines an equivalent norm to |\vlly.. 
H H Va 


Proof. The norm ||-||, defines an equivalent norm thanks to symmetry, continuity, and 
coercivity of a(., -; y). 
Since e(u) € Vp, from (1.53) with v, = e(p) it follows that 


Jean, = aCe), es) = r(e: y) < Ir Dl legy, 


the last inequality being due to the definition of the norm in V}. Furthermore, due to 
coercivity, we have 


[e| = ale), e); p) = ale], 
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Combining these two results yields (1.55). 
Furthermore, since l = f are linear forms, 


Sa (H) - Sy(H) = I(e(u); p) = f (e0); p) = a(Un), e0; p). (1.57) 
From (1.53) with v, := vy € Vy and (1.22) it follows that 
a(e(p), vy; H) = r(vy (H); p) = 0. 
This holds in particular for vy = uy(4). Moreover, due to symmetry, 
a(uy(H), e(p); p) = 0 
as well. Thus, a(u, (4), e(H); p) = a(e(p), e(H); p) in (1.57), and we conclude that 
ShD) - sr) = eol (1.58) 


The upper bound in (1.56) is then a consequence of (1.55), while the lower bound triv- 
ially holds as the right-hand side of (1.58) is a nonnegative quantity. 


Offline-online decomposition is usually solicited for the a posteriori error bounds 
introduced by the previous theorem, for the sake of a fast computation of the right- 
hand side of (1.55)—(1.56). This requires the efficient evaluation of both the numerator 
(dual norm of the residual) and the denominator (parameterized coercivity constant). 
The Riesz representation theorem is employed to define the unique r() € V, such that 


(wW, Va)y, =1(Vy3H), Yh € Vp. (1.59) 
Under affine separability assumptions (1.26)—(1.28), we have 
Qr h N Qa j 
rvw H) =} OWS) - DY Unn >, 8a4i(S" Vh) Wn € Vip 
i=l n=1 i=l 


so that an affine expansion with Q; + NQ, terms is obtained for r(-; u). Riesz represen- 
tation is then invoked for 


rV H) = fin)» eo Tans BD = fo, (Vn), 

TQ,+1(Vai H) = ay(5",Vn), sy Toa, VH) = dq, (67, Vn), 
N N 

TO-+(N-1)0,+1 (Vn M) = a,(¢" Vn)» tees TO-+NQ, Vn H) = ag, (F Vh) 


during the offline stage, storing the corresponding solutions to (1.59). 
As concerns the evaluation of the denominator of (1.55)-(1.56), exact evaluation 
of a(t) is seldom employed. Instead, an offline-online decomposable lower bound is 
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sought. Early proposals on the topic are available in [107, 78, 106, 85, 18]. In 2007, 
the successive constraint method (SCM) was devised in [57] based on successive linear 
programming approximations, and subsequently extended in [26, 27, 103, 111]. Alter- 
native methodologies based on interpolation techniques have also appeared in recent 
years in [54, 71, 59]. 

A posteriori error estimation can be derived for more general problems as well (in- 
cluding noncoercive linear, nonlinear, or time-dependent problems), through applica- 
tion of the Brezzi-Rappaz—Raviart theory. We refer to [106, 41, 109, 70, 81] for a few 
representative cases. To this end, extensions of SCM are discussed in [27, 55, 58, 25]. 


1.2 Geometrical parameterization for shapes and 
domains 


In this section we discuss problems characterized by a geometrical parameterization. 
In particular, a reference domain approach is discussed, relying on a map that de- 
forms the reference domain into the parameterized one. Indeed, while affine shape 
parameterization (see Section 1.1.3.5 for an example, and [85] for more details) natu- 
rally abides by the offline-online separability assumption, it often results in very lim- 
ited deformation of the reference domain, or strong assumptions on the underlying 
shape. 

Let Q c RÍ, d = 2,3, be the reference domain. Let M bea parametric shape mor- 
phing function, that is, 


M(x;p) : RÊ > RY, (1.60) 


which maps the reference domain Q into the deformed domain Q(u) as Q(u) = 
M(Q;u), where wp € P represents the vector of the geometrical parameters. This 
map will change accordingly to the chosen shape morphing technique. The case of 
Section 1.1.3.5 is representative of an affine map M(-; u). Instead, in the following we 
address more general (not necessarily affine) techniques such as FFD, RBF interpola- 
tion, and IDW interpolation. 

From a practical point of view, we recommend the Python package called PyGeM — 
Python Geometrical Morphing [79], which allows an easy integration with the majority 
of industrial CAD files and the most common mesh files. 


1.2.1 Free form deformation 


Free form deformation (FFD) is a widely used parameterization and morphing tech- 
nique both in academia and in industry. 
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For the original formulation see [94]. More recent works use FFD coupled with 
reduced basis methods for shape optimization and design of systems modeled by el- 
liptic PDEs (see [63], [86], and [96]), in naval engineering for the optimization of the 
bulbous bow shape of cruise ships (in [37]), in the context of sailing boats in [65], and 
in automotive engineering in [91]. 

FFD can be used for both global and local deformations and it is completely in- 
dependent of the geometry to morph. It acts through the displacement of a lattice of 
points, called FFD control points, constructed around the domain of interest. In par- 
ticular it consists in three different steps, as depicted in Figure 1.3. First the physical 
domain Q is mapped to Q, the reference one, through the affine map w. Then the lattice 
of control points is constructed, and the displacements of these points by the map T 
is what we call geometrical parameters u. The deformation is propagated to the entire 
embedded body usually by using Bernstein polynomials. Finally through the inverse 
map yp we return back to the parametric physical space Q(u). 


Figure 1.3: Scheme of the three maps 
composing the FFD map M. In par- 
ticular Y maps the physical space to 
the reference one, then 7 deforms 
the entire geometry according to the 
displacements of the lattice control 
points, and finally yt maps back the 
reference domain to the physical one. 


So, recalling equation (1.60), we have the explicit map for the FFD, that is, the 
composition of the three maps presented, i.e., 


M(x y) = (Wt ° T o p) p) = (1.61) 
L M N 

= w(> > $ bimn (POP? yn =) vx € Q, (1.62) 
l=0 m=0 n=0 


where bimn are Bernstein polynomials of degree l, m, nin each direction, respectively, 
and P? in(Hmn) = Pimn + Himn, With Pynn representing the coordinates of the control 
point identified by the three indices l, m, n in the lattice of FFD control points. In an 
offline-online fashion, for a given x, terms {bimn (W(X))}1m,n Can be precomputed during 
the offline stage, resulting in an inexpensive linear combination of x-dependent pre- 
computed quantities and w-dependent control points locations {P9 an (Himn) timn- The 
application of Y~ does not hinder such offline-online approach as w is affine. 


1 Basic ideas and tools for MOR of parametric PDEs —— 17 


We can notice that the deformation does not depend on the topology of the ob- 
ject to be morphed, so this technique is very versatile and nonintrusive, especially for 
complex geometries or in industrial contexts (see, e. g., [90, 87]). 

In the case where the deformation has to satisfy some constraints, like for example 
continuity constraints, it is possible to increase the number of control points. Often it 
is the case where at the interface between the undeformed portion of the geometry 
and the morphed area the continuity has to be prescribed for physical reasons. 

Asan example, in Figure 1.4 we present an FFD of a bulbous bow, where an STL file 
of a complete hull is morphed continuously by the displacement of only some control 
points. 


Figure 1.4: Bulbous bow deformation using FFD. In green are shown the FFD control points defining 
the morphing. 


1.2.2 Radial basis function interpolation 


Radial basis functions (RBFs) represent a powerful tool for nonlinear multivariate ap- 
proximation, interpolation between nonconforming meshes ([40]), and shape param- 
eterization due to their approximation properties [15]. 

An REF is any smooth real-valued function Ø : RÊ > R such thatg: R* > R 
exists and @(x) = (|x|), where || - || indicates the Euclidean norm in RÊ. The most 
widespread RBFs are the following: 

— Gaussian splines ([15]) defined as 


g(x) = eB, 


— thin plate splines ([42]) defined as 


g(ixl) = (HY in( Ht 


18 — G.Rozzaetal. 


— Beckert and Wendland C?-basis ([11]) defined as 


otal) = (1- ki (bal +1} 


-  multiquadratic biharmonic splines ([92]) defined as 


p(Ixll) = yilxll? + R?; 


- inverted multiquadratic biharmonic splines ([15]) defined as 


1 


Vix? +R? 


where R > Ois a given radius and the subscript , indicates the positive part. 
Following [75, 72], given Nç control points situated on the surface of the body to 
morph, we can generate a deformation by moving some of these points and impos- 
ing the new surface which interpolates them. The displacements of the control points 
represent the geometrical parameters 4. 
We can now define the map in equation (1.60) for the RBF interpolation tech- 
nique, that is, 


(xl) = 


Ne 
MEy) = 906 B) + Y Yi) (IX -xc,l), (1.63) 
i=1 
where q(x; y) is a polynomial term, generally of degree 1, y;(u) is the weight associated 
to the basis function 9;, {X¢, ye are control points selected by the user (denoted by 
spherical green markers in Figure 1.5), and x € Q. We underline that in the three- 
dimensional case (1.63) has dx Nç +d +d? unknowns, which are dx N¢ for y; and d+d* 
for the polynomial term q(x; y) = c(w)+Q(p)x. To this end we impose the interpolatory 
constraint 


MXc H) =¥c(M) Vie {1,...,Ne} (1.64) 


where yç, are the deformed control points obtained applying the displacement y to 
XCp in particular 


Xc = [Xa Xey 1 € RN, (1.65) 
yeH) = [Yc 0D, -< -Yc (W)] € RV. (1.66) 


For the remaining d + d* unknowns, due to the presence of the polynomial term, we 
complete the system with additional constraints that represent the conservation of the 
total force and momentum [15, 75] as follows: 


Ne 
$ vi) = 0, (1.67) 
i=1 
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Figure 1.5: Two different views of the same deformed carotid artery model using the RBF interpola- 
tion technique. The green dots indicate the RBF control points that define the morphing. The black 
small points highlight the original undeformed geometry. The occlusion of the two branches is 
achieved through a displacement along the normal direction with respect to the carotid surface of 
the control points after the bifurcation. 


Ne Ne 
È nWc h = 0... ¥ vil xola = 0, (1.68) 


i=1 i=1 
where the notation [x], denotes the d-th component of the vector x. 

Following an offline-online strategy, for a given x, evaluation of p(||x - X¢,|), i = 
1,...,. Nc, can be precomputed in the offline stage. Further online effort is only required 
for (i) given p, solve a dxNç+d +d’ linear system, and (ii) given pand x, perform linear 
combinations and the matrix vector product in (1.63) employing either precomputed 
quantities or coefficients from (i). 


1.2.3 Inverse distance weighting interpolation 


The Inverse distance weighting (IDW) interpolation method has been proposed in [95] 
to deal with interpolation of scattered data. We follow [108, 49, 9] for its presentation 
and the application of IDW to shape parameterization. 

As in the previous section, let Kg A S R be a set of control points. The IDW 
interpolant Imw(f) of a scalar function f : R? > R is defined as 


Ne 
Tw) = X wO f(xc,), xeQ, (1.69) 
k=1 
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where the weight functions wọ : Q — R, fork = 1,..., MN, are given by 


Ie-xg 
At xxe l k 


w) = 44 ifx =X, (1.70) 
(0) otherwise, 


where s is a positive integer, modeling the assumption that the influence of the k-th 
control point xç, on x diminishes with rate -s as the distance between x and xç, in- 
creases. IDW interpolation trivially extends to vector functions f : RÊ > RÊ by ap- 
plication to each component f;,...,f;, where the weight functions wọ : Q — R do not 
depend on the specific component. 

In the case of IDW shape parameterization, for any given p, the deformed position 
of the control points TARA is supposed to be known, and equal to yç, (4) := f Xc) 
for k = 1,..., No. We remark that the analytic expression of f is not known, but only 
its action through al This is indeed the minimum requirement to properly de- 
fine (1.69). The deformation map is therefore 


Ne 
MGH) = È WY H) Yx €Q. 
k=1 
In an offline-online separation effort, efficient deformation can be obtained by noting 
that the -dependent part is decoupled from the x-dependent weight function w;(x). 
Thus, for any x, weight terms can be precomputed once and for all and stored. The on- 
line cost of the evaluation of M(x; u) thus requires an inexpensive linear combination 
of x-dependent precomputed quantities and y-dependent control point locations. We 
remark that, in contrast, the RBF approach (even though still based on interpolation) 
required a further solution of linear system of size d x Nç + d + d’. 
Application in the context of fluid—structure interaction problems between a wing 
(structure) and surrounding air (fluid) is shown in Figure 1.6. The IDW deformation of 


Displacement 
4075e-01 


‘0.000e+00 


Figure 1.6: Deformation of the fluid mesh of a fluid—structure interaction problem by IDW. 
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the fluid mesh resulting from a vertical displacement of the tip of the wing is depicted; 
the structural mesh is omitted from the picture. We refer to [9] for more details. 


1.3 Beyond affinity assumptions: parametric 
interpolation 


We describe here several options to deal with cases when an exact affine decompo- 
sition of the discretized differential operators, right-hand sides, or outputs of interest 
does not exist. The section begins with a brief overview concerning the description 
of general nonaffine problems in Section 1.3.1 and later we describe the so-called EIM 
family of algorithms. This methodology becomes particularly useful to obtain an effi- 
cient offline-online splitting also in cases with nonlinearities and nonaffine parame- 
terization. We provide a full description of the different alternatives, starting from its 
standard continuous version (EIM), and presenting also its discrete (DEIM) and ma- 
trix (M-DEIM) variants. The methodologies are tested for both nonaffine and nonlinear 
problems. In Section 1.3.2 we explain in detail the basics of the EIM. In Section 1.3.3 
we introduce the discrete variant of the EIM at both matrix and vector level and we 
mention further options to obtain an approximate affine expansion. In Section 1.3.5 
we present two examples using the EIM (Section 1.3.5.1) and the M-DEIM algorithm to 
deal with both nonaffinity and nonlinearity (Section 1.3.5.2). 


1.3.1 Nonaffine problems 


As already discussed in Section 1.1.3.4, the existence of an affine decomposition of 
the linear and bilinear forms of the considered problem is crucial in order to obtain a 
computationally efficient framework (see (1.26)—(1.28)). 

This assumption fails to be true in several situations. Such situations occur for 
example in case of problems with nonaffine parametric dependency, in cases with 
nonlinear differential operators, and in cases dealing with the nonaffine geometrical 
parameterizations introduced in Section 1.2. 

In fact, in these situations, the differential operators, the right-hand sides, or the 
outputs of interest cannot be directly written using an exact affine decomposition and 
we have therefore to rely on an approximate affine decomposition. The EIM is one of 
the key instruments to recover an approximate affine decomposition. 

The EIM is a general tool for the approximation of parameterized or nonlinear 
functions by a sum of affine terms. In the following expression we report an example 
for a generic parameterized function f: 


Q 
FW = È cah). (1.71) 
q=1 
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The EIM has been firstly proposed in [10] to deal with nonaffine problems in the con- 
text of reduced basis methods and later applied to ROM in [53]. In [69] it has been 
extended to a general context, and a slightly different variant of EIM, DEIM, has been 
firstly proposed in [22, 23]. For more details on the a posteriori error analysis the inter- 
ested reader may see [53, 44, 24] while for an extension to hp-adaptive EIM we refer to 
[45]. A generalization of the EIM family of algorithms has been proposed in [68, 24, 67] 
while a nonintrusive EIM technique is presented in [21] and an extension with special 
focus on high-dimensional parameter spaces is given in [56]. 


1.3.2 The empirical interpolation method 


The EIM is a general method to approximate a parameterized function f (x;y) : Q x 
Prm — Rbya linear combination of Q precomputed basis functions in the case where 
each function f, := (+; 4) belongs to some Banach space Xo. In what follows u € Pry 
is the parameter vector and Pry is the parameter space. The EIM approximation is 
based on an interpolation operator Ig that interpolates the given function f, in a set 
of interpolation points x32, c€ Q. The interpolant function is constructed as a linear 
combination of hierarchically chosen basis functions hpg 1 € Vgm: Where Vg is an 
approximation of the function space // that contains f , i. e., Ygrm ¢ U. On the contrary 
to other interpolation methods, that usually work with generic and multipurpose basis 
functions such as polynomial functions, the EIM works with problem-specific basis 
functions with global support and selected hierarchically. The interpolant function 
can be then expressed by 


Q 
Iglfyl@) =} ch, xE, ME Prem (1.72) 
q=l 


where c} are parameter-dependent coefficients. Once the basis functions h(x) are set, 
the problem of finding the coefficients c,(q) is solved imposing the interpolation con- 
dition, i.e., 


Q 
flA) = È cgGDhg(Xq) = fiX) G=1....Q (1.73) 
q=1 


The above problem can be recast in matrix form as Tc, = f „ with 
(T)g=hx), (Gy), =, (FW), =fosm, iLj=1,...,Q. (1.74) 


This problem can be easily solved given the fact that the basis functions h,(x) and 
the interpolation points x, are known and that the matrix T is invertible. 
The selection of the basis functions Nghe, and of the interpolation points LAPE 


which are defined by a linear combination of selected function realizations Fadly 
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is done following a greedy approach similar to the one presented in Section 1.1.3.2 
(Algorithm 1.2). The procedure provides also a set of sample points MZ that are 
required for the construction of the basis functions. 

Since the basis functions are defined as linear combinations of the function re- 
alizations inside the parameter space, in order to approximate the function f with a 


relatively small number of basis functions h,, the manifold 


Mem = f; H) | U € Pem} (1.75) 


must have a small Kolmogorov N-width [61]. 

Once a proper norm on Q has been defined, where we consider L?(Q)-norms for 
1 < p < œ, the procedure starts with the selection of the first parameter sample, which 
is computed as 


Hy, = arg sup|f.| oa 
HEP am 


while the first interpolation point is computed as 
xX, =arg sup|f,, œ]. 
XEQ 


The first basis function and the interpolation operator at this stage are then defined 
as 


fu, 00 
f, A (x) i 


At the subsequent steps, the next basis function is selected as the one that is the worse 
approximated by the current interpolation operator and using a similar concept the 
interpolation point, often referred as magic point, is the one where the interpolation 
error is maximized. In mathematical terms, at the step k, the sample point is selected 
as the one that maximizes the error between the function f and the interpolation op- 
erator computed at the previous step I,_, [f]: 


hy (x) = L10) = f(x; Wh x). 


By = arg suphi - ka fOlio: 


HEPEmMm 


Once the sample point has been determined, the interpolation point is selected, in a 
similar fashion, as the point inside the domain that maximizes the error between the 
function f and the interpolation operator: 


Xç = arg sup lf, (x) — Ik- fap 100]. 


The next basis function is defined similarly to the first one with 


Fry. PO — Thay, 100 
fy (Xx) - Tha Up, 116 ) 


hy (x) = 
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Algorithm 1.2: The EIM algorithm — continuous version. 

Input: set of parameterized functions f, : Q — R, tolerance tol and maximum 
number of basis functions Nmax, P order of the chosen p-norm. 

Output: basis functions {h,,..., hg}, interpolation points {x,,...,Xg}; 
k=1;€ =tol+1; 
while k < Nmax and e > tol do 


Pick the sample point: 
Hx = arg suplifu œ) — Tkalfl OO llzecay; 
HEP am 


Compute the corresponding interpolation point: 
esate Sap Wy, 20) — Tk-a [fy JOO] 
XE 
Define a Dex past funcion: 
L a UO 
iy(x) = Fay, Mea y 1068)? 
Compute the error level: 
E= llEpllzce with E,(H) = Ifo) ~The fully; 
k=k+1; 
end while 


The procedure is repeated until a certain tolerance tol is reached or a maximum num- 
ber of terms Nmax are computed (Algorithm 1.2). We remark that by construction the 
basis functions {h;,...,4g} and the functions {f> -<> fu al span the same space Ypy: 


Vem = span{h,,..., ho} = span{fy»--- Sug} 


However, the former are preferred for the following reasons (for more details and for 

the mathematical proofs we refer to [10]): 

— they are linearly independent, 

- A(x;) =1forl<is< Qandh,(x;) = Oforl<i<j<Q, 

- they make the interpolation matrix T of equation (1.74) to be lower triangular and 
with diagonal elements equal to unity and therefore the matrix is invertible. 


The third point implies that the interpolation problem is well-posed. 


1.3.2.1 Error analysis 


Dealing with interpolation procedures, the error analysis usually involves a Lebesgue 
constant. In particular, in the case one is using the L°(Q)-norm the error analysis in- 
volves the computation of the Lebesgue constant A, = sup,<o phar |L,(x)| being L; € 
Vgm a Lagrange function that satisfies L;(x;) = 6,. It can be proved that the interpola- 
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tion error is bounded by the following expression [10]: 
fu = ly [fulllz(ay < (1 He Aq) inf lfa = Vgllz (a): (1.76) 
Vg€ Vem 


An upper bound for the Lebesgue constant, which in practice has been demonstrated 
to be very conservative [10], can be computed as 


q 
s2 -1 


For more details concerning the estimates of the interpolation error we refer to [10, 69]. 


1.3.2.2 Practical implementation of the algorithm 


Practically, finding the maximum of Algorithm 1.2 is usually not feasible and therefore 
the continuous version must be transformed into a computable one. 
This is done selecting a finite-dimensional set of training points in the parameter 


space {y,}¥, € PU c Phm and in the physical domain {x;}!@, € Q, c Q. For this 
reason we introduce the vector f : Qa x Pua" — R” which consists of a discrete 


representation of the function f: 
Fi =f.) t=1,...,M. (1.77) 


We also define the matrix Hg € RQ, which is defined by the discrete basis func- 
tions Hg = [h,,...,g] and the interpolation index vector ig = (i,,...,ig). The discrete 
interpolation operator of order Q for the vector function f is then defined by 


Ig ful — Hoar,» (1.78) 
where the coefficients ap are defined such that Taş, =f w where 
Tiq = (Ho)iq k,q=1,...,Q. (1.79) 


The implementation of the algorithm is similar to the continuous version and is re- 
ported in Algorithm 1.3. In the algorithm we use the notation F, ; to denote the j-th col- 
umn of the matrix F, where F € R“*" is a matrix containing vector representations of 
the function f: 


(F); = fX; M)).- (1.80) 


Once the basis and the interpolation indices are defined, during the online stage it is 
required to make a pointwise evaluation of the f function in the points defined by the 
interpolation indices. 
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Algorithm 1.3: The EIM algorithm — practical implementation. 


inpot: set of parameter samples {p}, € PHAN C Phm set of discrete points 


{x}; e otan, tolerance tol, maximum number of basis functions N nax p order of 
the chosen p-norm. 
Output: basis function matrix Hg = {h}, .. . , ho}, interpolation index vector 
ig = {i ...,ighs 
Assemble the matrix: 
(F)y=fOpe), t=1..5M, j=1,...,N; 
k=1, €=tol+1; 
while k < Nmax and £ > tol do 
Pick the sample index: 


Ik = arg mane, - Tal lilies 
j=1, 
and copie the interpolation point index: 


i, = arg max Fi, - (Ik [F. 5, Dil; 
i=l 
define he: next approximation column: 
h l,j, | 
k= Foy Te F Di, 

define the error level: 

e= max |F; - Ia lF jll 

k=k+1 
end while 


1.3.3 The discrete empirical interpolation method 


A computable version of EIM is the so-called DEIM, introduced in [23]. We provide here 
an example as a special case of EIM where appropriate basis functions are already 
available [69]. In our example here DEIM basis functions are computed relying on a 
POD procedure which is performed on a set of discrete snapshots of the parameterized 
function {f Te 1: Each snapshot f ; is already considered in discrete form in a prescribed 
set of points pe}. The procedure, which is described in detail in Algorithm 1.4, can 
be summarized into the following steps: 

1. Construct the DEIM basis functions using a POD procedure on a set of previously 

computed snapshots: 


Hy = [h ..., hy] = POD(f (Hp -- -> Hy). (1.81) 


2. Given a prescribed tolerance tol, determine the indices ig and truncate the dimen- 
sion of the POD space using an iterative greedy approach (Algorithm 1.4). 
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Algorithm 1.4: The DEIM procedure. 
Input: snapshots matrix S = [f(,),...;f(My,)], tolerance tol. 
Output: DEIM basis functions Hg = [h,,..., Mg], interpolation indices 
ig = [ip ..- iQ]. 
compute the DEIM modes Hy = [hy,..., hy] = POD(S) 
€=tol+1, k=1 
i = arg max |(h;)jl 
J=1,Nh 
Hg=(h], ig=[i), P= [e;] 
while £ > tol do 


k=k+1 
Solve (P”Ho)c = Phy, 
r= hy ie Hoc 


i, = arg max I(r); 
J=1,Np, 
Hg = [Hoy], P= [P,e;,]; ig = [lio ix] 


end while 


In Algorithm 1.4, with the term e;,, we identify a vector of dimension N, where the only 
nonnull element is equal to 1 and is located at the index i,: 


(e) =1forj= İk (e; )j = 0 forj # ix. 


During the online stage, when a new value of the parameter u needs to be tested, it 
is required to compute the function f(y) only in the location identified by the indices 
ig. Therefore, the nonlinear function needs to be evaluated only in a relatively small 
number of points which is usually much smaller with respect to the total number of 
degrees of freedom used to discretize the domain. 


1.3.4 Further options 


Apart from the EIM and the DEIM algorithm, further options are available. We mention 
here the matrix version of the DEIM algorithm (M-DEIM) [14], which extends the DEIM 
also to the case of parameterized or nonlinear matrices, the generalized EIM (GEIM) 
[68], and the gappy POD [16, 20]. 

The M-DEIM is used to perform MOR on discretized differential operators charac- 
terized by nonlinearity or nonaffinity with respect to the parameter vector u. The al- 
gorithm is similar to the one in Algorithm 1.4 with the only difference that a vectorized 
version of the matrices is used to describe snapshots and POD modes. In Section 1.3.5 
we will provide an example dealing with both issues. 
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The gappy POD generalizes the interpolation condition to the case where the 
number of basis functions is smaller than the number of interpolation indices, i.e., 
card(H) < card(ig). In this case the interpolation condition is substituted by a least- 
squares regression. 

The GEIM replaces the EIM requirement of a pointwise interpolation condition by 
the following statement: 


oj(Ig(f(H))) =9(FQ@), j=1...0 (1.82) 


where o; are a set of “well-chosen” linear functionals. For more details and for conver- 
gence analysis of the present method we refer to [67]. 


1.3.5 Some examples 


In the previous sections we have presented the EIM family of algorithms and we have 
illustrated how it is possible to recover an approximate affine expansion of the dis- 
cretized differential operators. In this section we show in more detail two examples 
on the practical application of the EIM and the M-DEIM algorithm. 


1.3.5.1 A heat transfer problem with a parameterized nonaffine dependency forcing 
term 


In this example we illustrate the application of the computable version of the EIM 
on a steady-state heat conduction problem in a two-dimensional square domain Q = 
[-1,1]* with a parameterized forcing term g(u) and homogeneous Dirichlet boundary 
conditions on the boundary ðQ. The problem is described by the following equation: 


E =g), inQ, as 


0=0, onðQ, 


where 0 is the temperature field, a, is the thermal conductivity coefficient, and g(p) is 
the parameterized forcing term which is described by the following expression: 


gp) = e720) -2007p (1.84) 


where u and y, are the first and second components of the parameter vector and x, 
and x, are the horizontal and vertical coordinates, respectively. Let V be a Hilbert 
space. The weak formulation of the problem can be written as follows: Find 0 € V 
such that 


a(O(p),v;H) =fv;y) WweV, (1.85) 
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where the parameterized bilinear and linear forms are expressed by 


a(O, v; p) = | ve -Vvdx, f(v;p) = | sccwvax, (1.86) 
a a 


In the above expressions, the bilinear form a(.,-;p) : Vx V — Ris trivially affine while 
for the linear form f (;u) : V — R we have to rely on an approximate affine expan- 
sion using the EIM. The problem is discretized using triangular linear finite elements 
according to the mesh reported on the left side of Figure 1.9. 

In the present case it is not possible to write an exact affine decomposition of the 
linear form f; we rely therefore on the computable version of the EIM of Algorithm 1.3 
in order to recover an approximate affine expansion. 

The function g(x; u) is parameterized with the parameter vector y = (44,42) € 
Prem = [-1L 1]?, which describes the position of the center of the Gaussian function. 
The conductivity coefficient a, is fixed constant and equal to 1. The testing set for the 
implementation of the algorithm {;};-y,.,. € Ptrain İS defined using Nyain = 100 and 
a uniform probability distribution. The set of points CANA € Q that is used for the 
identification of the magic points is chosen to be coincident with the nodes of the fi- 
nite element grid reported in Figure 1.7. In Figure 1.8 we report the first four EIM basis 
functions for the nonlinear function g and the location of the magic points identified 
by the EIM algorithm. In Figure 1.9 we report the convergence analysis of the EIM al- 
gorithm for the nonlinear function g changing the number of EIM basis functions (left 
plot) and the convergence analysis of the ROM changing the number of reduced basis 
functions (right plot). 


0 
0.0 0.085 0.178 


Figure 1.7: Discretized domain into which the parameterized problem is solved (left image), together 
with an example of the value assumed by the temperature field for one particular sample point in- 
side the parameter space (right image). 
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0. a es 1.00e+00 


Figure 1.8: Plot of the first four modes identified by the EIM algorithm (first row and left image in 
the second row) and the location of the first 35 indices ig. The magic points are identified by the red 
elements in the right picture on the second row. 


10° 


107 10" | 


Average of the relative error 
Average of the relative error 


0 5 10 15 20 25 30 35 2 5 7 10 12 15 17 20 
N of EIM bases N of basis functions 

Figure 1.9: Convergence analysis of the numerical example. In the left plot we can see the average 
value of the L? relative error between the exact function g and its EIM approximation. On the right 
plot we report the average value of the L? relative error between the FOM temperature field and the 
ROM temperature field. The plot is for different numbers of basis functions used to approximate 
the temperature field and keeping constant the number of basis functions used to approximate the 
forcing term (N = 11). 
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1.3.5.2 An example in the context of reduced-order models with nonlinearity and 
nonaffine parametric dependency 


In this second illustrative example we show the application of the DEIM algorithm to 
the stationary parameterized Navier-Stokes equations. In the present case we have 
both nonlinearity and nonaffinity with respect to the input parameters. Both nonlin- 
earity and nonaffinity have been tackled using the matrix version of the DEIM. The 
computational domain is given by the unit square Q = [0,1]? and the physical prob- 
lem is described by the well-known Navier-Stokes equations: 


div(u $ u) — div(2v(qu)V5u) = -Vp, inQ, 


divu = 0, in Q, 

(1.87) 
u(x) = (1,0), on Trop, 
u(x) = 0, onTo. 


The physical problem is the classical benchmark of the lid-driven cavity problem with 
a parameterized diffusivity constant v(y). In this case the impossibility of recovering 
an affine decomposition of the differential operators is given by the convective term, 
which is by nature a nonlinear term, and by the parameterized diffusion term. The 
diffusivity constant v(u) has in fact been parameterized by the following nonlinear 
function: 


@2(-2(%-Hy-0.5)"-20“Hp-0.5)”) 


V(X; H) = 00 + 0.01, (1.88) 


which is a Gaussian function and the position of whose center has been parameter- 
ized using the parameter vector 4 = (4, 42). For the particular case, the discretized 
algebraic version of the continuous formulation can be rewritten as 


Go r) (“) 7 (5). (1.89) 


The matrix A(p) represents the discretized diffusion operator, the matrix C(u) repre- 
sents the discretized nonlinear convective operator, while the term B represents the 
divergence operator. The term A(y) is characterized by a nonaffine parametric depen- 
dency while the term C(u) is characterized by nonlinearity with respect to the solution. 
The velocity and pressure fields are approximated as 


Ny Np 
u) = > oh, p) = > Eh, (1.90) 
q=1 q=1 


and, in order to achieve an efficient offline-online splitting, the discretized operators 
are approximated by the matrix version of the DEIM algorithm and expressed as 


Ny Ne 
Ap) = X Gwh, Cu) = > ch(e, hy. (1.91) 
q=1 q=1 
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The problem is discretized using the finite volume method and a staggered Carte- 
sian grid made of 20 x 20 cell-centered finite volume elements. The DEIM algorithm 
has been implemented using 100 samples chosen randomly inside the training space 
prain € [-0.5,0.5]*. The magic points necessary for the implementation of the DEIM 
algorithm are chosen to be coincident with the cell centers of the discretized problem. 
The basis functions h and hi are obtained using the DEIM algorithm applied on the 
vectorized version of the discretized differential operator snapshots computed during 
the training stage S4 = [vec(A,),...,vec(Ajy)] and Sç = [vec(C;),...,vec(Cm)]. The 
snapshot matrices S4 and Sç contain in fact the discretized differential operators in 
vector form obtained for the different samples of the training set. 

In Figure 1.10 we report the comparison of the full-order model fields and the ROM 
ones; the comparison is depicted for a parameter sample not used to train the ROM. 


U Magnitude Ured Magnitude 
0.00e+00 0.20.30.40.50.60.70.8 1.00e+00 0.00e+00 0.20.30.40.50.60.70.8 1.00e+00 
L ji | | 
(a) (b) 


-4.20e-01 0 0.2 0.4 7.70e-01 -4.20e-01 0 0.2 0.4 7.70e-01 
1 | | | 
(c) (d) 


Figure 1.10: Comparison between the FOM velocity (a) and pressure (c) fields and the ROM velocity 
(b) and pressure (d) fields. The plots are reported for one selected sample value inside the testing 
set. The ROM solutions have been computed using 14 basis functions for the velocity space, 10 for 
the pressure space, 10 DEIM basis functions for the convective matrix C, and 10 DEIM basis functions 
for the diffusion matrix B. 
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Figure 1.11: Eigenvalue decay of the POD procedure during the DEIM algorithm (left plot). The con- 
vergence analysis with respect to the number of DEIM basis functions (right plot), which is com- 
puted using the average value over the testing set of the L? relative error, has been performed keep- 
ing constant the number of basis functions used to approximate the velocity and pressure fields 

(N, = 14, Ny = 10) and changing the number of DEIM basis functions used to approximate the con- 
vective and diffusion terms (Nc = N4). 


On the right side of Figure 1.11 we report the convergence analysis for the numerical 
example. The plots are performed testing the ROM on 100 additional sample values se- 
lected randomly inside the parameter space Pity, € [-0.5, 0.5]°. In the plots is reported 
the average value over the testing space of the L? relative error. 


1.4 Advanced tools: reduction in parameter spaces 


Often the use of the aforementioned geometrical morphing techniques in Section 1.2 
does not tell us how many control points, i. e., geometrical parameters, are enough 
to conduct a proper analysis. This leads to self-imposing too few parameters in order 
to avoid the curse of dimensionality and dealing with intractable problems. To over- 
come this issue there exist techniques for parameter space dimensionality reduction, 
both linear and nonlinear. In particular we present here the active subspaces property 
for linear dimensionality reduction, while in the last section we show an overview of 
possible nonlinear methods. 

These methods are intended as general tools, not restricted to parameterized 
PDEs. Moreover, the nature of the parameter space can be very diverse, including 
both geometrical and physical parameters. They are data-driven tools working with 
couples of input/output data, and they can be used to enhance other MOR techniques. 


1.4.1 Active subspaces property and its applications 


In this and the following sections we present the active subspaces (AS) property pro- 
posed by Trent Russi [89] and developed by Paul Constantine [31]. In brief, active sub- 
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spaces are defined as the leading eigenspaces of the second moment matrix of the 
function’s gradient and constitute a global sensitivity index. 

We present how to exploit AS to reduce the parameter space dimensionality, and 
use it as a powerful preprocessing tool. Moreover, we show how to combine it with 
a model reduction methodology and present its application to a cardiovascular prob- 
lem. In particular, after identifying a lower-dimensional parameter subspace, we sam- 
ple it to apply further MOR methods. This results in improved computational effi- 
ciency. 

The main characteristic of AS is the fact that it uses information of both the out- 
put function of interest and the input parameter space in order to reduce its dimen- 
sionality. The active subspaces have been successfully employed in many engineering 
fields. We cite, among others, applications in magnetohydrodynamics power genera- 
tion modeling in [50], in naval engineering for the computation of the total drag resis- 
tance with both geometrical and physical parameters in [101, 38], and in constrained 
shape optimization [66] using the concept of shared active subspaces in [100]. There 
are also applications to turbomachinery in [7], to uncertainty quantification in the nu- 
merical simulation of a scramjet in [34], and to the acceleration of Markov chain Monte 
Carlo in [35]. Extension of active subspace discovery for time-dependent processes 
and application to a lithium ion battery model can be found in [32]. A multifidelity 
approach to reduce the cost of performing dimension reduction through the compu- 
tation of the active subspace matrix is presented in [62]. In [46] the authors exploit AS 
for Bayesian optimization, while the coupling with ROMs can be found in [39] for a 
nonintrusive data-driven approach, and the coupling with POD-Galerkin methods for 
biomedical engineering will be presented in Section 1.4.4 following [99]. 


1.4.2 Active subspaces definition 


Given a parametric scalar function f(u) : RP? — R, where p is the number of pa- 
rameters representing the output of interest, and given a probability density function 
p : R? — R* that represents uncertainty in the model inputs, active subspaces are 
low-dimensional subspaces of the input space where f varies the most on average. It 
is a property of the pair (f, p) [31]. In order to uncover AS we exploit the gradients of 
the function with respect to the input parameters, so it can be viewed as a derivative- 
based sensitivity analysis that unveils low-dimensional parameterization of f using 
some linear combinations of the original parameters. Roughly speaking, after a rescal- 
ing of the input parameter space to the hypercube [-1, 1]”, we rotate it until the lower- 
rank approximation of the output of interest is discovered, which means a preferred 
direction in the input space is identified. Then we can project all the data onto the or- 
thogonal space of this preferred direction and we can construct a surrogate model on 
this low-dimensional space. 
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Let us add some hypotheses to f in order to properly construct the matrix we will 
use to find the active subspaces: Let f be continuous and differentiable with square- 
integrable partial derivatives in the support of p. We define the so-called uncentered 
covariance matrix C of the gradients of f as the matrix whose elements are the average 
products of partial derivatives of the map f, that is, 


C= EWS If] = [OAO pdy, (1.92) 
where E is the expected value and V,f = Vf(H) = ie eae a is the column vector 
P 


of partial derivatives of f. This matrix is symmetric so it has a real eigenvalue decom- 
position: 


C = WAW', (1.93) 


where W € R”® is the orthogonal matrix of eigenvectors and A is the diagonal ma- 
trix of nonnegative eigenvalues arranged in descending order. The eigenpairs of the 
uncentered covariance matrix define the active subspaces of the pair (f, p). Moreover, 
Lemma 2.1 in [33] states that the eigenpairs are functionals of f (u) and we have 


A; = w7 Cw; = | ow Twi) p dy, (1.94) 


which means that the i-th eigenvalue is the average squared directional derivative of f 
along the eigenvector w;. Alternatively we can say that the eigenvalues represent the 
magnitude of the variance of V,f along their eigenvectors orientations. So small val- 
ues of the eigenvalues correspond to small perturbation of f along the corresponding 
eigenvectors. It also follows that large gaps between eigenvalues indicate directions 
where f changes the most on average. Since we consider the lower-dimensional space 
of dimension M < p where the target function has exactly this property, we define the 
active subspace of dimension M as the span of the first M eigenvectors (they corre- 
spond to the most energetic eigenvalues before a gap). Let us partition A and W as 


A, 
a=| a 


where A, = diag(A,,...,A,;) and W, contains the first M eigenvectors. We can use W, to 
project the original parameters to the active subspace obtaining the reduced param- 
eters, that is, the input space is geometrically transformed and aligned with W,, in 
order to retain only the directions where the function variability is high. We call the 
active variable py the range of W; and the inactive variable 7 the range of W;: 


W=[W, W], (1.95) 


By =W? e R", n= Wye RP, (1.96) 
We can thus express any point in the parameter space yu € R” in terms of wy and y as 


p= WW’ p= W,Wi p+ W,W7y = Wipy + Won. (1.97) 
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The lower-dimensional approximation, or surrogate quantity of interest, g 
R” — R of the target function f is a function of only the active variable Hy as 


fW = g(W7H) = 2m). (1.98) 


Such g is called ridge function [77] and, as we can infer from this section, it is constant 
along the span of W). 

From a practical point of view, equation (1.92) is estimated through the Monte 
Carlo method. We draw Nyain independent samples p® according to the measure p 
and we approximate 


Norain 
c=¢=—~— Y vf Vf! = WAW’, (1.99) 
Merain i=1 


where Vifi = Vaf qu). In [31] the authors provide a heuristic formula for the number 
of samples N rain needed to properly estimate the first k eigenvalues, that is, 


Nerain = &k In(p), (1.100) 


where a usually is between 2 and 10. Moreover, they prove that for sufficiently large 
N rain the error € committed in the approximation of the active subspace of dimension 
nis bounded from above by 


2 Aâ 
€ = dist(rank(W,), rank(W;)) < ae 
An = Ant 


(1.101) 
where 6 is a positive scalar bounded from above by ae Here we can clearly see 
how the gap between two eigenvalues is important in order to properly approximate f 
exploiting AS. 


1.4.3 Some examples 


In this section we present two simple examples with the computation of the active 
subspaces using analytical gradients. To highlight the possibility that the presence of 
an active subspace is not always guaranteed we also show an example in this direc- 
tion. We choose for both the cases a three-dimensional input parameter space without 
loss of generality. In order to identify the low-dimensional structure of the function of 
interest we use the sufficient summary plots, developed in [36]. In our cases, they are 
scatter plots of f (u) against the active variable wy. 

The presence of an active subspace is not always guaranteed. For example, not 
every target function that has a radial symmetry has a lower-dimensional representa- 
tion in terms of active variables. This is due to the fact that there is no rotation of the 
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input parameter space that aligns it along a preferred direction since all of them are 
equally important. 

Let us consider for example the function f (u) = 5 p'u representing an n-dimen- 
sional elliptic paraboloid, where the parameter p is a column vector in [-1,1]?. In this 
case we have the exact derivatives, in fact, V,f = y, and we do not have to approximate 
them. If we draw 1,000 samples and we apply the procedure to find an active subspace 
and we plot the sufficient summary plot in one dimension, as in Figure 1.12, we clearly 
see how itis unable to find the active variable along whichf varies the most on average. 
In fact there is not a significant gap between the eigenvalues, since we have C = iId. 
Moreover the projection of the data onto the inactive subspace suggests the presence 
of an n-dimensional elliptic paraboloid. 


Eigenvalues 


2 3 45 40 05 OO 05 10 «15 
Index Active variable Wy 


Figure 1.12: Example of an output function with a radial symmetry. On the left the exact eigenvalues 
of the uncentered covariance matrix are shown. On the right the sufficient summary plot in one di- 
mension (f(u) against wy = wip) shows how the projection of the data along the inactive directions 
does not unveil a lower-dimensional structure for f. 


Let us consider now another quadratic function in three variables. We define the out- 
put of interest f as 


fp) = ; py’ Ap, (1.102) 


where u € [-1, 1)? and A is symmetric positive definite with a major gap between the 
first and the second eigenvalue. With this form we can compute the exact gradients as 
Vaf (p) = A and, taking p as a uniform density function, compute C as 


C= a( f pup dnja" = a (1.103) 
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So the squared eigenvalues of A are the eigenvalues of C. Since, by definition, A has a 
significant gap between the first and second eigenvalues, we can easily find an active 
subspace of dimension one. 

In Figure 1.13 we show the sufficient summary plot of f with respect to its active 
variable. A clear univariate behavior is present, as expected, so we can easily con- 
struct g, for instance taking a quadratic one-dimensional function. We can also see 
the associated eigenvalues of the uncentered covariance matrix. 
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Figure 1.13: Example of a quadratic function with an active subspace of dimension one. On the left 
the exact eigenvalues of the uncentered covariance matrix are shown. On the right the sufficient 
summary plot in one dimension (f (u) against py = wip) shows how the projection of the data along 
the inactive directions unveils a univariate structure for f. 


1.4.4 Active subspaces as preprocessing tool to enhance model 
reduction 


The presence of an active subspace for an output of interest, derived from the solution 
of a parametric PDE, can be exploited for further MOR. Thus, in this context, AS can 
be seen as a powerful preprocessing technique to both reduce the parameter space 
dimensionality and boost the performance of other model order reduction methods. 

In [99] the active subspace for a relative pressure drop in a stenosed carotid artery 
is used as areduced sampling space to improve the reconstruction of the output man- 
ifold. We used as parameters the displacement of a selection of RBF control points 
to simulate the occlusion of the carotid artery after the bifurcation. For a review of 
the RBF interpolation technique, see Section 1.2.2. In Figure 1.5 two different views of 
the same carotid are shown and the control points are highlighted with green dots. 
The target function was a relative pressure drop between the two branches computed 
solving a stationary Navier-Stokes problem. 
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After the identification of the active subspace we exploit it by sampling the orig- 
inal full parameter space along the active subspace. These sampled parameters were 
used, in the offline phase, to construct the snapshots matrix for the training of an 
ROM. This leads to better approximation properties for a given number of snapshots 
with respect to usual sampling techniques. The natural construction of the uncentered 
covariance matrix, which uses information from both the inputs and the outputs, is 
the reason of such improvements. 

The same idea has been coupled also with nonintrusive MOR techniques, such as 
POD with interpolation (PODI), in [102], while for the reconstruction of modal coeffi- 
cients using PODI with AS for low computational budgets we suggest [39]. 


1.4.5 About nonlinear dimensionality reduction 


There are plenty of other techniques that reduce the dimensionality of a given data 
set. They do not exploit simultaneously the structure of the output function and the 
input parameter space like AS, they just express the data vectors we want to reduce in 
a reduced space embedded in the original one. For a comprehensive overview, see [64] 
and [104]. The main assumption is that the data set at hand has an intrinsic dimen- 
sionality, which is lower than that of the full space where they belong. This means 
that the data are lying on or near a manifold with dimensionality d embedded in a 
greater space of dimension D. If we approximate this manifold with a linear subspace 
we use a linear dimensionality reduction technique; otherwise assuming the data lie 
on a curved manifold we can achieve better results using a nonlinear method. Un- 
fortunately in general neither the characteristics of the manifold, nor the intrinsic di- 
mensionality are known, so the dimensionality reduction problem is ill-posed. There 
are several algorithms to detect the intrinsic dimensionality of a data set; we suggest 
the review in [17]. Among all we cite two of the most popular techniques, i. e., locally 
linear embedding (LLE), presented in [83], and Isomap [98]. Extensions for the two 
methods can be found in [12]. 

LLE seeks to preserve local properties of the high-dimensional data in the em- 
bedded space, and it is able to detect nonconvex manifolds. In particular it preserves 
local reconstruction weights of the neighborhood graph, that is, LLE fits a hyperplane 
through each data point and its nearest neighbors. Some applications can be found 
in [51] for biomedical engineering, or in [60] for computational mechanics. 

Isomap instead seeks to preserve geodesic (or curvilinear) distances between the 
high-dimensional data points and the lower-dimensional embedded ones. Its topo- 
logical stability has been investigated in [8], while it has been used for micromotility 
reconstruction in [5]. 

Other approaches include for example a manifold walking algorithm that has 
been proposed in [73] and in [74]. 
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1.5 Conclusion and outlook 


This introductory chapter provided the means to understand projection-based MOR 
methods in Section 1.1. Various techniques allowing the parameterization of compli- 
cated geometries were provided in Section 1.2. Since many geometries of interest in- 
troduce nonlinearities or nonaffine parameter dependency, an intermediate step such 
as the EIM is often applied. The basics were presented in Section 1.3 and will be used 
further in the chapter on hyperreduction (Chapter 5) of this volume. The reduction in 
parameter space becomes necessary if high-dimensional parameter spaces are con- 
sidered. Active subspaces (Section 1.4) provide a mean to tackle the curse of dimen- 
sionality. 

Each chapter of the handbook gives in-depth technical details upon a particular 
topic of interest. This includes common MOR methods, several application areas of 
interest, and a survey of current software frameworks for model reduction. Whenever 
a method does not rely only on the PDE-based functional analysis setting introduced 
in this chapter, corresponding requirements will be mentioned within each technical 
chapter. 
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2 Model order reduction by proper orthogonal 
decomposition 


Abstract: We provide an introduction to proper orthogonal decomposition (POD) 
model order reduction with focus on (nonlinear) parametric partial differential equa- 
tions (PDEs) and (nonlinear) time-dependent PDEs, and PDE-constrained optimiza- 
tion with POD surrogate models as application. We cover the relation of POD and 
singular value decomposition, POD from the infinite-dimensional perspective, re- 
duction of nonlinearities, certification with a priori and a posteriori error estimates, 
spatial and temporal adaptivity, input dependency of the POD surrogate model, POD 
basis update strategies in optimal control with surrogate models, and sketch related 
algorithmic frameworks. The perspective of the method is demonstrated with several 
numerical examples. 


Keywords: POD model order reduction, (discrete) empirical interpolation, adaptivity, 
parametric PDEs, evolutionary PDEs, certification with error analysis 
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2.1 Introduction 


Proper orthogonal decomposition (POD) is a method which comprises the essential 
information contained in data sets. Data sets may have their origin in various sources, 
like, e. g., (uncertain) measurements of geophysical processes, numerical simulations 
of (parameter-dependent) complex physical problems, or (dynamical) imaging. In or- 
der to illustrate the POD idea of information extraction, let {y,,...,y,} c R” denote a 
vector cloud (which here serves as our data set), where we suppose at least one of the 
vectors y; is nonzero. Let us collect the vectors y; in the data matrix 


Y = [y,|..-ly,] € R”. 


Then we have r = rank Y e€ {1,..., min(m, n)}. Our aim now is to find a vector p € R” 
with length one which carries as much information of this vector cloud as possible. 
Of course, we here have to specify what information in this context means. For this 
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purpose we equip R” with some inner product (-,-) and induced norm ||- ||. We define 
the information content of vector y with respect to some unit vector y by the quantity 
Iy, w)|. Then we determine the special vector y € R” by solving the maximization 
problem 


pe asma op) |W € R” with ||| = i} (2.1) 
jal 


Note that the solution to the maximization problem in (2.1) is not unique. If b is a 
vector, where the maximum is attained, then -y is an optimal solution, too. Let us 


label the vector Ų by w,. We now iterate this procedure; suppose that for 2 < £ < r 


we have already computed such £ — 1 orthonormal vectors wy and then seek a unit 


vector Y; € R” which is perpendicular to the (£ — 1)-dimensional subspace 
vi = span{y,,...,e4}¢ R”, 


and which carries as much information of our vector cloud as possible, i.e., satisfies 


Wp = argmax} 510) p |W € R” with ||] = 1and Y 1 val, 


j=l 
It is now straightforward to see that the vectors {w,;}i_, are given by 
Wp, =; 1<i<r, (2.2) 
where the prs solve the eigenvalue problem (cf. [42, 53]) 
VV" wb; =A; i=1,...,r andà >- >A, > 0, 
where Ÿ = W"?Y e R™®” with the symmetric, positive definite (weighting) matrix 
W = (((e5 &)) Jii jem (2.3) 


In (2.3) the vector e; denotes the i-th unit vector in R™. The modes (wie, obtained 
in this way are called POD modes or principal components of our data cloud. If now 
m > n > r, itis advantageous to consider the eigenvalue problem 


Y' Yq; = Api i=1,...,randA, =--->A,>0, 


which admits the same eigenvalues A; as before. The modes y; and @;, i = 1,...,r, are 
related by singular value decomposition (SVD): 


p; = Y¢;, he eee 


1 
Oi 
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and g; = yA; > 0 is the i-th singular value of the weighted data matrix Y. Note that in 
contrast to (2.2) the square root matrix Ww"? is not required. 

It is now clear that a vector cloud also could be replaced by a function cloud 
{y(Hj) lj = 1,...,n} c X in some Hilbert space (X, (-,-)), where {uh are parameters 
which may refer to, e. g., time instances of a dynamic process, or stochastic variables, 
and the concept of information extraction by the above maximization problems di- 
rectly carries over to this situation. As shown in the next section, we can even extend 
this concept to general Hilbert spaces. This will be formalized in Section 2.2.1. From 
the considerations above it also becomes clear that POD is closely related to SVD. This 
is outlined in Section 2.2.2. The POD method for abstract nonlinear evolution prob- 
lems is explained in Section 2.2.3. The Hilbert space perspective also allows us to treat 
spatially discrete evolution equations, which include adaptive concepts for the spa- 
tial discretization. This is outlined in Section 2.2.4. The POD-Galerkin procedure is ex- 
plained in Section 2.3, including a discussion of the treatment of nonlinearities. The 
certification of the POD method with a priori and a posteriori error bounds is outlined 
in Section 2.4. The POD approach heavily relies on the choice of the snapshots. Related 
approaches are discussed in Section 2.5. In Section 2.6 we briefly address the scope 
of the POD method in the context of optimal control of partial differential equations 
(PDEs). Finally, in Section 2.7 we sketch further important research trends related to 
POD. Our analytical exposition is supported by several numerical experiments which 
give an impression of the power of the approach. 

POD is one of the most successfully used model reduction techniques for non- 
linear dynamical systems; see, e. g., [23, 42, 53, 75, 90] and the references therein. It 
is applied in a variety of fields, including fluid dynamics, coherent structures [4, 9], 
and inverse problems [13]. Moreover, in [11] POD is successfully applied to compute 
reduced-order controllers. The relationship between POD and balancing was consid- 
ered in [61, 82, 100]. An error analysis for nonlinear dynamical systems in finite di- 
mensions was carried out in [78] and a missing point estimation in models described 
by POD was studied in [10]. 


2.2 POD 


In this section we introduce a discrete variant of the POD method, where we follow 
partially [42, Section 1.2.1]. For a continuous variant of the POD method and its rela- 
tionship to the discrete one we refer the reader to [58] and [42, Sections 1.2.2 and 1.2.3]. 


2.2.1 The POD method 


Suppose that K, n4, ..., ng are fixed natural numbers. Let the so-called snapshot en- 
sembles wi, c X be given for 1 < k < K, where X is a separable real Hilbert space. 
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For POD in complex Hilbert spaces we refer the reader to [96]. We set n = n; +--+- + ng. 
To avoid a trivial case we suppose that at least one of the y;’s is nonzero. Then we 
introduce the finite-dimensional, linear snapshot space 


V = span {yj |1<j<mandi<k<K}cX (2.4) 


with finite dimension d < n. We distinguish two cases: 

1) Theseparable Hilbert space X has finite dimension m: Then X is isomorphic to R”; 
see, e. g., [81, p. 47]. We define the finite index set I = {1,...,m}. Clearly, we have 
1 < r < min(n,m). Especially in the case of X = R”, the snapshots yy = (Vi) iciem 
are vectors in R” for k =1,...,K. 

2) X is infinite-dimensional: Since X is separable, each orthonormal basis of X has 
countably many elements. In this case X is isomorphic to the set £, of sequences 
{Xj}ie~ Of complex numbers which satisfy °°, Ixi? < 00; see [81, p. 47], for in- 
stance. The index set I is now the countable but infinite set N. 


The POD method consists in choosing a complete orthonormal basis {w;};-; in X such 
that for every £ € {1,...,r} the information content of the given snapshots yy is maxi- 
mized in the following sense: 


SH yk 2 
max» > )% |i) x 
i=1 k=1 j=l 


s.t. {Pihi CX and iP) = dj, 1<ijse 


(P*) 


with positive weighting parameters ai ,j=1,...,n,andk =1,...,K. Here, the symbol 
ôi; denotes the Kronecker symbol satisfying ô; = 1 and 6; = 0 fori + j. 

An optimal solution CARR to (P) is called a POD basis of rank £. It is proved in 
[42, Theorem 1.8] that for every £ € {1,...,r} a solution {Pi} to (P*) is characterized 
by the eigenvalue problem 


RY; = AVY; for 1 < i < £, (2.5) 


where À; > --- > À, > O denote the largest eigenvalues of the linear, bounded, nonneg- 
ative, and self-adjoint operator R : X — X given as 


K nk 
RY = ya (wh) yf for Y e X. (2.6) 
k=1j=1 


Moreover, the operator R can be presented in the form 
R=yy" (2.7) 
with the mapping 


K Mm —__ 
y:R" >X, yo) =X X yak Ky for ® = (j,...,h,) € R”, 
kija 
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where y* : X — R” denotes the Hilbert space adjoint of Y, whose action is given by 


yi) = (CY, Vetyt) ys Y, aK yK yy) for Y e X. 


The operator K : R” > R”, K := Y*y then admits the same nonzero eigenvalues 
A, 2 ++- 2A, > O with corresponding eigenvectors ®},...,®,, and its action is given by 


K nk 


KD = Y D faro, VD o VERO OYn) 2.8) 
k=1j=1 


with the vector ® = (@j,... Pr.) € R”. For the eigensystems of R and K we have the 


relation 
@, = as yy, and Y= ae 
l IK; l l T 


Furthermore, we obtain 


yO; fori=1,...,r. (2.9) 


= 


k 


© K k k 2 £ 
Za lY Ey = A 
i=l k= jal i=l 


l 


~ 


and for the POD projection error we get 


2 


K n aly £ K m% 7 r 
DLG- LODY = Yo w (2.10) 
k=1j=1 i=1 k=1 j=1 xX i 


Thus, the decay rate of the positive eigenvalues {A;};_, plays an essential role for a 
successful application of the POD method. In general, one has to utilize a complete 
orthonormal basis {¥;};-, € X to represent elements in the snapshot space V by their 
Fourier sum. This leads to a high-dimensional or even infinite-dimensional approx- 
imation scheme. Nevertheless, if the term )'}_,,, A; is sufficiently small for a not too 
large £, elements in the subspace V can be approximated by a linear combination of 
the few basis elements (PIE. This offers the chance to reduce the number of terms in 
the Fourier series using the POD basis of rank £, as shown in the following examples. 
For this reason it is useful to define information content of the basis {¥;}{_, in V by the 
quantity 


iA; 
Èi-14; 
It can, e. g., be utilized to determine a basis of length £ € {1,..., r} containing = 99 % 


of the information contained in V by requiring £(£) ~ 99%. Now it is shown in [42, 
Section 1.2.1] that 


E(€) = € [0, 1]. (2.11) 


r K i H2 
2 A= bap a; ly; lz 
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holds true. This implies 


vad 


eel Kn ak ky? 
Èk diet a; ly; lz 


so that the quantity £ (£) can be computed without knowing the eigenvalues A,,,,...,A,. 


2.2.2 SVD and POD 


To investigate the relationship between SVD and POD, let us discuss the POD method 
for the specific case X = R”. Then we define the matrices 


ay 0) 
pK = a e R™™ forl<k<K, 
k 
0) An, 
D! 0 
D = ., € Re 
o DĚ 


y“ = [yil yn] eR™™ forl<k<K, 
Ys[V [a(R row" ore, 


where we have introduced the weighting matrix W € R™™ in (2.3). 


Remark 2.1. Let us mention that Y = Y holds true provided all ai: are equal to one 
G. e., D is the identity matrix) and the inner product in X is given by the Euclidean 
inner product (i.e., W is the identity matrix). 


Now (2.5) is equivalent to the m x m eigenvalue problem 
YY'W, =A¥; forl<i<eé (2.12) 
with ¥; = WP; and the n x n eigenvalue problem 
Y'Y@, =A,®, forl<i<é (2.13) 


with Y; = YD" ?®;/ VÄ. If m < n holds, we solve (2.12). However, we have to solve the 
linear system WY; = Ẹ; for anyi = 1,..., £in order to get the POD basis {¥,}{_,. Thus, 
ifn < mholds, we will compute the solution CAH to (2.13) and get the POD basis by 
the formula ¥; = YD"?®;/ VÄ. In that case we also have Y'Y = Y'WY so that we 
do not have to compute the square root matrix WwW’. On the other hand, the diagonal 
matrix D"? can be computed easily. The relationship between (2.12) and (2.13) is given 
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by SVD: There exist real numbers g; > --- > ø, > 0 and orthogonal matrices Y € R™”, 
® € R”” with column vectors {¥;}!",, {Di}, respectively, such that 


Ty = 0 mxn 
Y Y= =LeR™, (2.14) 
o O 
where ¥' = diag (0;,...,0,) € R?“ and the zeros in (2.14) denote matrices of appropri- 
ate dimensions. Moreover, the vectors {;}i_, and {®;}!_, are eigenvectors of YY' and 
Y'Y, respectively, with eigenvalues A; = (0;)* > 0 fori = 1,...,r. The vectors {¥;}7",,, 
and {®;}"_,,, (ifr < m respectively r < n) are eigenvectors of YY' and Y'Y with eigen- 
value 0. We summarize the computation of the POD basis in the pseudo-code function 


CY, A]=POD(Y, W, D, £, flag). 


function [¥, A]=PODCY, W, D, £, flag) 
Require: Snapshots matrix Y = [Y!,..., Y] with rank r, weighting matrices W, D, 
number £ of POD functions, and flag for the solver; 
1: if flag = 0 then 
2 Set ¥ = wi?yp"?; 
3: Compute singular value decomposition [V, £, ®] = svd(Y); 
4: Define Ẹ; as the i-th column of Y and g; = Za forl<i< £; 
5: Set ¥; = WP; and A; = 0? fori =1,...,8; 
6: else if flag = 1 then 
7: 
8 
9 


Compute eigenvalue decomposition [Y, A] = eig (YY"); 
Define ¥; as the i-th column of Y and A; = Aj for 1 < i< £; 
Set Y; = WY; fori =1,..., 6; 

10: else if flag = 2 then 

1: Compute eigenvalue decomposition [®, A] = eig(Y'Y); 

12: Define @; as the i-th column of ® and À; = A; for1 < i < £; 

13: Set Y; = YD? h;/ VĂ fori =1,...,8; 

14: endif 

15: return Y = [¥;|...|¥;] and A = [A |... [Ae]; 


2.2.3 The POD method for nonlinear evolution problems 


In this subsection we explain the POD method for abstract nonlinear evolution prob- 
lems. We focus on the numerical realization. For detailed theoretical investigations we 
refer the reader to [42, 50, 51, 57, 58], for instance. 
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2.2.3.1 Nonlinear evolution problems 


Let us formulate the nonlinear evolution problem. For that purpose we suppose the 
following hypotheses. 


Assumption 2.1. Suppose that T > O holds, where [0, T] is the considered finite time 

horizon. 

1) Vand H are real, separable Hilbert spaces and suppose that V is dense in H with 
compact embedding. By <- ,-) and (-,-)y we denote the inner products in H and V, 
respectively. We identify H with its dual (Hilbert) space H’ by the Riesz isomorphism 
so that we have the Gelfand triple 


VoH=H' OV’, 


where each embedding is continuous and dense. The last embedding is understood 
as follows: For every elementh € H' andv € V, we also havev < H by the embedding 
V — H, so we can define (h',v) yy = (h',V) py. 
2) Foralmost allt € [0, T] we define a time-dependent bilinear form a(t;-,-): Vx V —> 
R satisfying 
la(ts 9, @)| < yllpllylldlly forallg,p €V, teé[0,Tlae,  (2.15a) 
ats9.~) > yı lply -yzlplz forall € V, t € [0,T]a.e.,  (2.15b) 


for time-independent constants y, y, = 0, y, > O and where “a. e.” stands for “almost 
everywhere”. 

3) Assume that y, € V,f < L°(0,T;H) holds. Here we refer to [27, pp. 469-472] for 
vector-valued function spaces. 


Recall the function space 
W(0, T) = {ọ € L°(0,T; V) |p; € L (0, T; V')}, 


which is a Hilbert space endowed with the standard inner product; cf. [27, pp. 472- 
479]. Furthermore, we have 


£ (Q(t), 9) = (Ot), P) yy forg € W(0,T), pe V 


in the sense of distributions in [0, T]. Here, (-,-)y:,y stands for the dual pairing be- 
tween V and its dual V’. 

Now the evolution problem is given as follows: Find the state y € W(0,T) N 
C([0, T]; V) such that 


d 
a VO Pn + Ab YO.9) + NVO) 9) vv = FOP) 
vy € V,t € (0,T]a.e., (2.16) 
(YO), P)y = V> Py Vp €H. 


2 Model order reduction by proper orthogonal decomposition —— 55 


Throughout we assume that (2.16) admits a unique solution y € W(0,T) n 
C([0, T]; V). Of course, this requires some properties for the nonlinear mapping NV 
which we will not specify here. 


Example 2.1 (Semi-linear heat equation). Let Q c RÍ, d € {2,3}, be a bounded open 
domain with Lipschitz-continuous boundary ðQ and let T > O be a fixed end time. 
We set Q := (0,T) x Q and È := (0,T) x OQ and c = O. For a given forcing term f € 
L?(Q) and initial condition Y, € L?(Q), we consider the semi-linear heat equation with 
homogeneous Dirichlet boundary condition: 


y(t, xX) — Ay(t, X) + cy’ (t,x) = f(t,x) ing, 
y(t,x) =0 on X, (2.17) 
y(0,x)=y.(x) inQ. 


The existence of a unique solution to (2.17) is proved in [83], for example. We can 
write (2.17) as an abstract evolution problem of type (2.16) by deriving a variational 
formulation for (2.17) with V = Hy(Q) as the space of test functions H = LQ) and 
integrating over the space Q. The bilinear form a: V x V > Ris introduced by 


alp, p) = [vo voa foro, € V 
Q 


and the operator V : V > V’ is defined as N (p) = cy? for ọ € V. For c = 0, the heat 
equation (2.17) is linear. 


Example 2.2 (Cahn-Hilliard equations). Let Q, T, Q and È be defined as in Exam- 
ple 2.1. The Cahn -Hilliard system was proposed in [21] as a model for phase separation 
in binary alloys. Introducing the chemical potential w, the Cahn-Hilliard equations 
can be formulated in the common setting as a coupled system for the phase field c 
and the chemical potential w: 


c(t, xX) +y- Ve(t,x) = mAw(t, x) in Q, 
w(t,x) = —oeAc(t,x) + T w' (c(t, x)) inQ 
E (2.18) 
Vc(t,X) -Vo = Vw(t,X) -Vvo =0 on È, 
c(0, x) = c,(X) in Q. 


By vo we denote the outward normal on ðQ, m > O is a constant mobility, 0 > O 
denotes the surface tension, and 0 < € « 1 represents the interface parameter. Note 
that the convective term y - Vc describes the transport with (constant) velocity y. The 
transport term represents the coupling to the Navier-Stokes equations in the context 
of multiphase flow; see, e. g., [52] and [2]. The phase field function c describes the 
phase of a binary material with components A and B and takes the values c = —1 in 
the pure A-phase and c = +1 in the pure B-phase. The interfacial region is described 
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by c € (-1,1) and admits a thickness of order O(€); see, e. g., Figure 2.5, left column, 
where the binary phases are colored in blue and green, respectively, and the interfacial 
region is depicted in white. The function W(c) represents the free energy and is of 
double well-type. A typical choice for W is the polynomial free energy function 


W?(c) = (1- è} /4 (2.19) 


with two minima at c = +1, which describe the energetically favorable states. It is 
infinitely often differentiable. Another choice for W is the C! relaxed double obstacle 
free energy 


1 
an) = 50 c’) + = (max(c 1,0)? + min(c + 1,0)’), (2.20) 


with relaxation parameter s > 0, which is introduced in [43] as the Moreau—Yosida 
relaxation of the double obstacle free energy 


W(c) = ie =), ifc e [-1,1], 
+00, otherwise. 

The energies W?” (c) and wrl(c) later will be used to compare the performance of 
POD on systems with smooth and less smooth nonlinearities. For more details on the 
choices for W we refer to [1] and [19], for example. Concerning existence, uniqueness, 
and regularity of a solution to (2.18), we refer to [19]. In order to derive a variational 
form of type (2.16), we rewrite (2.18) as a single fourth-order parabolic equation for c 
by 


c(t,x) +y - Ve(t,x) = ma( -oec + <w" (cle x)) in Q, 


0 = Velt, x): vo = v(-oeact, X) + Zw (et)) -Vo ond, (2.21) 


c(0,X) = c. (X) in Q. 


We choose V = {v € H'(Q) : a fay = 0} equipped with the inner product (u, v)y := 
fa VuVv, so that the dual space of V is given by V’ = {f € (H'(Q))' : (f, 1) = 0} such that 
V > H = V' and (.,.) denotes the duality pairing. We note that (V, (., .)y) is a Hilbert 
space. We define the V’-inner product for f,g € V' as (f, g)y := Ío V-A) tf - V(-A) tg 
where (-A)™! denotes the inverse of the negative Laplacian with zero Neumann bound- 
ary data. Note that (f,g)y = (f,(-A)"g);2(q) = (AT, 8). We introduce the bi- 
linear forma: V x V > R by 


1 
a(u, v) = o€(Vu, Vv) 12a) + me - Vu, v)y' 
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and define the nonlinear operator M by N (c) = ew! (c). The evolution problem can 
be written in the form 

~(c4(t).v)y +a(c(t),v) + (N(c(t)),v) =O Wv eV anda.a. t e (0,T].’ (2.22) 
We note that this fits our abstract setting formulated in (2.16) with the Gelfand triple 
VoHeV'oV'. 


2.2.3.2 Temporal discretization and the POD method 


Let0 = tı < +- < ta, = T bea given time grid with step sizes At; = t;—G_, forj = 2,...,m. 
Suppose that for any j € {1,...,n,} the element y; € V c H is an approximation of y(t;) 
computed by applying a temporal integration method (e. g., the implicit Euler method) 
to (2.16). Then we consider the snapshot ensemble 


V=span{yj|1<j<n}cVcH 


with n = n; and r = dim V < n. In the context of (P°) we choose K = 1 and n = n} = ne. 
Moreover, X can be either V or H. For the weighting parameters we take the trapezoidal 
weights 


At; + At;_ A 
seti, a; = = II forj =2,...,n,—1, a, = . (2.23) 


J ny 


Of course, other quadrature weights are also possible. Now, instead of (P) we consider 
the maximization problem 


Coin 
2 
max a; Iy; Yi) 
22 Op FI (2.24) 


s.t. {Wi}, CX and (Y, Y) = 6, 1sijse 


ip 
with either X = V or X = H. 


Remark 2.2. In [42, Sections 1.2.2 and 1.3.2] a continuous variant of the POD method 
is considered. In that case the trapezoidal approximation in (2.24) is replaced by inte- 
grals over the time interval [0, T]. More precisely, we consider 


max 


l 


s. t. {Wi}. c X and (Y, Y) =ô 


IVO, ¥;) x at 


£ 
=1 


(2.25) 


s omn 


p isijse 


1 We acknowledge a hint of Harald Garcke who pointed this form of the weak formulation of (2.21) to 
us. 
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with either X = V or X = H. For the relationship between solutions to (2.24) and (2.25) 
we refer to [58] and [42, Section 1.2.3]. 


To compute the POD basis (Pi, of rank £ we have to evaluate the inner products 
(Yj Yi)x, where either X = V or X = H holds. In typical applications the space X is 
usually infinite-dimensional. Therefore, a discretization of X is required in order to 
get a POD method that can be realized on a computer. This is the topic of the next 
subsection. 


2.2.3.3 Galerkin discretization 


We discretize the state equation by applying any spatial approximation scheme. Let 
us consider here a Galerkin scheme for (2.16). For this reason we are given linearly 
independent elements ,,...,@,, € V and define the m-dimensional subspace 


V’ = span {P1 ..-; Øm} C V 
endowed with the V topology. Then a Galerkin scheme for (2.16) is given as follows: 
Find y” € W(0, T) n C([0, T]; V") satisfying 
T ONO, a)y + alsy'o.9") + WOO) Pvy 
= (f(),0"), Yo" €V", te (0,Tlae., (2.26) 
(©), 9"), = Yoy vote vt. 


Inserting the representation y"(t) = yey (O9; eV", t € [0, T], in (2.26) and choosing 
g" = ~; for i = 1,...,m we derive the following m-dimensional initial value problem: 


MY”) + A(oy"(t) + N"(y"(t)) = F’) for t € (0, T], ee 
M”y”(0) = yh, i 


where we have used the matrices and vectors 


M’ = (P; Pid) icijenv y"(t) = (ices, fort € [0, T] a.e., 
A(t) = (a(t; pj» Pi)) )i<ijem yn = (Yo Pi) Htciem? 


N"(v) = ((( 2P.»y) er) ) for v = (Vj)i<jem> 
V',V / 1<i<m 


F'(t) = (FOP )rciem for t € [0, T]. 


In the pseudo-code function [Y]=StateSol(y”) we present a solution method 
for (2.27) using the implicit Euler method. 

In the next subsection we discuss how a POD basis EA of rank £ < r can be 
computed from numerical approximations for the solution y” to (2.27). 
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function [Y]=StateSol(y” ) 


Require: Initial condition y’; 
1: Compute y? € R” solving M’y! = y!; 
2: forj = 2 to n, do 
3: Set A} = A(t) e€ R™™ and Fi = F"(t) e R”; 
4: Solve (e. g., by applying Newton’s method) for y e R” 


h hsh hy h hon h 
(M° + AtA; Wy; + AGN (y;) = M'y;_, + AGF;; 


5: end for 
6: return matrix Y = [y} EAA e R™", 


2.2.3.4 POD method for the fully discretized nonlinear evolution problem 


Recall that we have introduced the temporal grid A c [0,T] and set n = n,. Let 
ył, es syn € V” be numerical approximations to the solution y"(t) to (2.27) at time 
instances t = tis j = 1...,m,. Then, a coefficient matrix Y € R”*" is defined by the 
elements Y; given by 


m 
y’ = > Vii eV" forl<j<n. 
i=l 
The j-th column of Y (denoted by y; = Y.;) contains the Galerkin coefficients of the 
snapshot y? € V”. We set r = rank Y < min(m, n) and 


V" = span {y |1 <j <n} c v”, 


Due to V" c V" we have Y; € V” for 1 < j < £. Therefore, there exists a coefficient 
matrix Y € R! that is defined by the elements Y; satisfying 
m 
Y=} Yjp eV” forl<j<é@, 
i=1 


where the j-th column Y. j of the matrix Y consists of the Galerkin coefficients of the 
element ¥;. Note that 


ww") = (v")' "we, Ww"), = (v")'stw" 


hold for v” = Di v'Q;, w” = YS wto; € V" and for the symmetric, positive definite 


stiffness matrix 


st = (KP; Pidy) icijem 
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Then, we have for X = H 

ViVi) = y MY; = YIM'Y.; forl<j<n1<i<é@, 
and for X = V 

OP Yy = y S'Y; = yis'y,; forl<j<ni<i<é. 


Thus, we can apply the approach presented in Section 2.2.2choosing W = M” for X = H 
and W = S} for X = V. Moreover, we set K = 1,n, =n, = n, and aj =a; defined in (2.23). 
Now a POD basis of rank £ for (2.27) can be computed by the pseudo-code function 
LY,WJ=Popstate(y", W, D, £, flag ). 


function [Y, Y]=PoDState(y”, W, D, £, flag ) 


Require: Initial condition y”, weighting matrices W, D, number £ of POD functions, 
and flag for the solver; 
1: Call [Y]=StateSol(y"); 
2: Call C¥,A]=POD(Y, W, D, £, flag); 
3: return Y = [y] |...|y}, ]and Y = [¥;|...| Wel; 


In the next subsection we will discuss in detail how the POD method has to be 
applied in that case if we have — instead of V” — different spaces V" foreach PSA, uns 


2.2.4 The POD method with snapshots generated by spatially 
adaptive finite element methods 


In practical applications it often is desirable to provide POD models for time-de- 
pendent PDE systems, whose numerical treatment requires adaptive numerical tech- 
niques in space and/or time. Snapshots generated by those methods are not directly 
amenable to the POD procedure described in Section 2.2.3.4, since the application of 
spatial adaptivity means that the snapshots at each time instance may have different 
lengths due to their different spatial resolutions. In fact, there is not one single dis- 
crete Galerkin space V” for all snapshots generated by the fully discrete evolution, but 
at every time instance t; the adaptive procedure generates a discrete Galerkin space 


v’s c X, so that in this case y? = y’ € V", For this reason, no snapshot matrix Y can 
be formed with columns containing the basis coefficient vectors of the snapshots. 

To obtain also a POD basis in this situation we inspect the operator K of (2.8) and 
observe that its action can be computed if the inner products yy 5 y! )x can be evaluated 
foralll<i<n,1<sj<n,andi<kl<K. 
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Let us next demonstrate how to compute a POD basis for snapshots residing in 
arbitrary finite element (FE) spaces. To begin with we drop the superindex h and set 
V; = V’. For each time instant j = 1,...,n of our time-discrete PDE system the snap- 
shots Yj are taken from different finite element spaces V; £ X (j = 1, ..., n), where X 
denotes a common (real) Hilbert space. Let V; = span t, oe Pn) Then we have the 
expansions 


=) yg; forj=1,....n (2.28) 
i=1 


with coefficient vectors 
yj=(yj))€R”™ forj=i,...,n 


containing the finite element coefficients. The inner product of the associated func- 
tions can thus be computed as 


Vir Yj) 25 Syy Y (Yo), forij=1....n, 


k=1 [=1 


so that the evaluation of the action K® only relies on the evaluation of the inner prod- 
ucts (gh, VAS, Q<ij<s<nisk<m,1l<i< m;). In other words, once we are 
able to compute those inner products we are in the position to set up the eigensystem 
{(A;, D) }i-1 of K from (2.8). The POD modes {¥;};_, can then be computed according 
to (2.9) by 


Y, = PA fori=1,...,r 
Details on this procedure can be found in [62, 39]. 
To illustrate how this procedure can be implemented we summarize Examples 6.1- 
6.3 from [39], which deal with nested and nonnested meshes. All coding was done in 
C++ with FEniCS [8, 66] for the solution of the differential equations and ALBERTA 
[87] for dealing with hierarchical meshes. The numerical tests were run on a compute 
server with 512GB RAM. 


Run 2.1 ([39, Example 6.1]). We consider Example 2.1 with homogeneous Dirichlet 
boundary condition and vanishing nonlinearity, i.e., we set c = O so that the equa- 
tion becomes linear. The spatial domain is chosen as Q = (0,1) x (0,1) c R2, the 
time interval is [0, T] = [0, 1.57]. Furthermore, we choose X = L?(Q). For the temporal 
discretization we introduce the uniform time grid by 


j= (G-At forj=1,...,m = 1571 
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with At = 0.001. For the spatial discretization we use h-adapted piecewise linear, con- 
tinuous finite elements on hierarchical and nested meshes. Snapshots of the analyti- 
cal solution at three different time points are shown in Figure 2.1. Details on the con- 
struction of the analytical solution and the corresponding right-hand side f are given 
in [39, Example 6.1]. 


Figure 2.1: Run 2.1. Surface plot (top) and view from above (bottom) of the analytical solution of (2.17) 
at t = t, (left), t = T/2 (middle), and t = T (right). 


Due to the steep gradients in the neighborhoods of the minimum and the maximum, 
the use of an adaptive finite element discretization is justified. The resulting compu- 
tational meshes as well as the corresponding finest mesh (reference mesh at the end 
of the simulation which is the union of all adaptive meshes generated during the sim- 
ulation) are shown in Figure 2.2. 


Figure 2.2: Run 2.1. Adaptive finite element meshes att = t (left), t = T/2 (middle left), and t = T 
(middle right), and finest mesh (right). 


The number of nodes of the adaptive meshes varies between 3,637 and 7,071 points. 
The finest mesh has 18,628 degrees of freedom. A uniform mesh with grid size of or- 
der of the diameter of the smallest triangles in the adaptive grids (hmin = 0.0047) 
would have 93,025 degrees of freedom. This clearly reveals the benefit of using adap- 
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tive meshes for snapshot generation, which is also well reflected in the comparison 
of the computational times needed for the snapshot generation on the adaptive mesh 
taking 944 seconds compared to 8,808 seconds on the uniform mesh (see Table 2.4 for 
the speedup factors obtained by spatial adaptation). In Figure 2.3, the resulting nor- 
malized eigenspectrum of the correlation matrix K is shown for snapshots obtained by 
uniform spatial discretization (“uniform FE mesh”), for snapshots obtained by inter- 
polation on the finest mesh (“adaptive FE mesh”), and for snapshots without interpo- 
lation (“infPOD”), where K is associated to the operator K from (2.8); see also (2.30). 


adaptive FE mesh 
uniform FE mesh 

adaptive FE mesh, bigger error tol 
- - -adaptive FE mesh, smaller error tol 


|—adaptive FE mesh| |—adaptive FE mesh] 
|—uniform FE mesh |—uniform FE mesh 
+ infPOD 


+ infPOD 


A, / trace 


0 500 1000 1500 0 50 100 150 200 0 50 100 150 200 
index i index i index i 


Figure 2.3: Run 2.1. Comparison of the normalized eigenvalues using an adaptive and a uniform spa- 
tial mesh, respectively. Left: all eigenvalues; middle: first 200 largest eigenvalues; right: first 200 
largest eigenvalues with different error tolerances for the adaptivity (1.5 times bigger and smaller 
error tolerances, respectively). 


We observe that the eigenvalues for both adaptive approaches coincide. This numer- 
ically validates what we expect from theory: The information content which is con- 
tained in the matrix K when we explicitly compute the entries without interpolation 
is the same as the information content contained within the eigenvalue problem which 
is formulated when using the finest mesh. No information is added or lost. Moreover, 
we recognize that about the first 28 eigenvalues computed corresponding to the adap- 
tive simulation coincide with the simulation on a uniform mesh. From index 29 on, 
the methods deliver different results: For the uniform discretizations, the normalized 
eigenvalues fall below machine precision at around index 100 and stagnate. On the 
contrary, the normalized eigenvalues for both adaptive approaches flatten in the order 
around 10~?°. If the error tolerance for the spatial discretization error is chosen larger 
(or smaller), the stagnation of the eigenvalues in the adaptive method takes place at a 
higher (or lower) order (Figure 2.3, right). Concerning dynamical systems, the magni- 
tude of the eigenvalue corresponds to the characteristic properties of the underlying 
dynamical system: the larger the eigenvalue, the more information is contained in the 
corresponding eigenfunction. Since all adaptive meshes are contained in the uniform 
mesh, the difference in the amplitude of the eigenvalues is due to the interpolation 
errors during refinement and coarsening. This is the price we have to pay for faster 
snapshot generation using adaptive methods. A further aspect gained from the decay 
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behavior of the eigenvalues in the adaptive case is the following: The adaptive ap- 
proach filters out the noise in the system which is related to the modes corresponding 
to the singular values that are not matched by the eigenvalues of the adaptive ap- 
proach. This in the language of frequencies means that the overtones in the systems 
which get lost in the adaptive computations live in the space which is neglected by 
the POD method based on adaptive finite element snapshots. From this point of view, 
adaptivity can be interpreted as a smoother. 

The first, second, and fifth POD modes of Run 2.1 obtained by the adaptive ap- 
proach are depicted in Figure 2.4. We observe the classical appearance of the basis 
functions. The initial condition is reflected by the first POD basis function. The next 
basis functions admit a number of minima and maxima corresponding to the index in 
the basis: ¥, has two minima and two maxima, etc. This behavior is similar to the in- 
creasing oscillations in higher frequencies in trigonometric approximations. The POD 
basis functions corresponding to the uniform spatial discretization have a similar ap- 
pearance. 


Figure 2.4: Run 2.1. Surface plot (top) and view from above (bottom) of the POD basis functions Y, 
(left), W, (middle), and Y, (right). 


Run 2.2 ([39, Example 6.2]). (Cahn-Hilliard system). We consider Example 2.2 in the 
form (2.18) with Q = (0, 1.5) x (0, 0.75), T = 0.025, constant mobility m = 0.00002, and 
constant surface tension o = 24.5. The interface parameter € is set to € = 0.02, with 
resulting interface thickness 7 - € ~ 0.0628. We use the relaxed double obstacle free 
energy we from (2.20) with s = 10%. As initial condition, we choose a circle with ra- 
dius r = 0.25 and center (0.375, 0.375). The initial condition is transported horizontally 
with constant velocity v = (30, o)!. We set 


ti=(j-1)At forj=1,...,n, = 1001, 
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so that At = 2.5 - 10°. The numerical computations are performed with the semi- 
implicit Euler scheme. For this purpose let d! € V andc’ e V denote the time- 
discrete solution at t;_, and t;, respectively. Based on the variational formulation (2.22) 
we tackle the time discrete version of (2.18) in the following form: Given co find d, 
w solving 


a G = O71) 2 +(v- vo, P1) 2 +m (vw, V1) 22 =0, 
hak | be eas | (2.29) 
—(w’, @) 72 + 0€ (VE, Vo) 2 + z (Wi (d) + W! (0), P2) =0 


for all p}, € V andj = 2,...,n, with c! = c,. According to (2.22), here it is V = 
{v € H'(Q), ial fo vdx = 0}. Note that the free energy function W is split into a convex 
part W, and a concave part W_, such that W = W, + W_ and W! is treated implicitly, 
whereas W’ is treated explicitly with respect to time. This leads to an unconditionally 
energy-stable time marching scheme; compare [33]. The system (2.29) is discretized in 
space using piecewise linear and continuous finite elements. The resulting nonlinear 
equation systems are solved using a semi-smooth Newton method. 

Figure 2.5 shows the phase field (left) and the chemical potential (right) for the fi- 
nite element simulation using adaptive meshes. The initial condition c, is transported 
horizontally with constant velocity. 


Figure 2.5: Run 2.2. Phase field c (left) and chemical potential w (right) computed on adaptive finite 
element meshes at t = t (top), t = T/2 (middle), and t = T (bottom). 


The adaptive finite element meshes and the finest mesh which is generated during the 
adaptive finite element simulation are shown in Figure 2.6. The number of degrees of 
freedom in the adaptive meshes varies between 6,113 and 8,795. The finest mesh (over- 
lay of all adaptive meshes) has 54,108 degrees of freedom, whereas a uniform mesh 
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Figure 2.6: Run 2.2. Adaptive finite element meshes att = t (top left), t = 7/2 (top right), and t = T 
(bottom left) together with the finest mesh (bottom right). 


with discretization fineness as small as the smallest triangle in the adaptive meshes 
has 88,450 degrees of freedom. 

Figure 2.7 shows the first, second, and fifth POD modes for the phase field c and 
the chemical potential w. Analogously to Run 2.1, we observe a periodicity in the POD 
basis functions corresponding to their basis index numbers. 


Figure 2.7: Run 2.2. First, second, and fifth POD modes for c (left) and w (right). 
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In the present example we only compare the POD procedure for two kinds of snap- 
shot discretizations, namely, the adaptive approach with using a finest mesh and the 
uniform mesh approach, where the grid size is chosen to be of the same size as the 
smallest triangle in the adaptive meshes. We choose X = L7(Q) and compute a sepa- 
rate POD basis for each of the variables c and w. 

In Figure 2.8, a comparison is visualized concerning the normalized eigenspec- 
trum for the phase field c and the chemical potential w using uniform and adaptive 
finite element discretization. We note for the phase field c that about the first 180 eigen- 
values computed corresponding to the adaptive simulation coincide with the eigenval- 
ues of the simulation on the finest mesh. Then, the eigenvalues corresponding to the 
uniform simulation decay faster. Similar observations apply for the chemical poten- 
tial w. 


10° 10° 


— adaptive FE mesh| — adaptive FE mesh 


— uniform FE mesh — uniform FE mesh 
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index i index i 


1015 
0 


Figure 2.8: Run 2.2. Comparison of the normalized eigenvalues for the phase field c (left) and the 
chemical potential w (right) using an adaptive and a uniform spatial mesh, respectively. 


We use the criterion (2.11) to determine the basis length £ which is required to represent 
a prescribed information content with the respective POD space. We will choose the 
POD basis length £, for the phase field c and the number of POD modes £, for the 
chemical potential, such that 


€min = arg min {€(2): E(@)>1-p}, with £ = £, and £p, respectively, 


for a given value p representing the loss of information. Alternatively, the POD basis 
length could be chosen in alignment with the POD projection error (2.10) with the ex- 
pected spatial and/or temporal discretization error; compare, e. g., [39, Theorem 5.1]. 
Let us also refer, e. g., to the recent paper [12], where different adaptive POD basis ex- 
tension techniques are discussed. Table 2.1 summarizes how to choose £, and ¢,, in 
order to capture a desired amount of information. Moreover, it tabulates the POD pro- 
jection error (2.10) depending on the POD basis length, where Aj and A;" denote the 
eigenvalues for the phase field c and the chemical potential w, respectively. The re- 
sults in Table 2.1 agree with our expectations: the smaller the loss of information p is, 
the more POD modes are needed and the smaller is the POD projection error. 
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Table 2.1: Run 2.2. Number of needed POD bases in order to achieve a loss of information below 
the tolerance p using adaptive finite element meshes (columns 2-5) and uniform finite element 
discretization (columns 6-9) and POD projection error. 


p e Live ay ead Live ay’ emi Lise A; gmi Lise ay, 
10? 3 20-10% 4 156.9-10° 3 20-10% 4 157.6-10° 
107 10 21-10% 13 15.8-10° 10 21-104 13 15.6-10° 
103 19 25-10°° 26 1.8:10° 19 2.5.10 25 1.8-10° 
104 29 20-10 211 4118-107 28 26-10% 160 1.9-107% 
10°° 37 25.107 644 11-107 37 24-107 419 2.5.10? 


Run 2.3 ([39, Example 6.3]). (Linear heat equation revisited). We again consider Ex- 
ample 2.1 with c = 0. The purpose of this example is to confirm that our POD ap- 
proach also is applicable in the case of nonnested meshes like it appears in the case 
of r-adaptivity, for example. We set up the matrix K for snapshots generated on se- 
quences of nonnested spatial discretizations. This requires the integration over cut 
elements; see [39]. We choose Q = (0,1) x (0,1) c R°, [0,T] = [0,1], and we apply a 
uniform temporal discretization with time step size At = 0.01. The analytical solution 
in the present example is given by 


y(t,xX) = sin(7X9) - sin(71x,) - cos(27tx9), 


with x = (Xo,X,), source term f := y, — Ay, and the initial condition g := y(0,-). The 
initial condition is discretized using piecewise linear and continuous finite elements 
ona uniform spatial mesh, which is shown in Figure 2.9 (left). Then, at each time step, 
the mesh is disturbed by relocating each mesh node according to the assignment 


Xo = Xp t+O-XQ- (Xp —1)- (At/10), 
x = xX +0-05-x,- (x, -1)- (At/10), 


where 0 € R, is sufficiently small such that all coordinates of the interior nodes ful- 
fillO < x) < 1and O < x, < 1. After relocating the mesh nodes, the heat equation 
is solved on this mesh for the next time instance. We use Lagrange interpolation to 
transfer the finite element solution of the previous time step onto the new mesh. The 
disturbed meshes at t = 0.5 and t = 1.0 and an overlap of two meshes are shown in 
Figure 2.9. To compute the matrix K from (2.30) we have to evaluate the corresponding 
inner products of the snapshots, where we need to integrate over cut elements. 

We compute the eigenvalue decomposition of the matrix representation K of the 
operator K (cf. (2.30)) for different values of 0 and compare the results with a uniform 
mesh (i. e., 0 = 0) in Figure 2.10. We note that the eigenvalues of the disturbed mesh are 
converging to the eigenvalues of the uniform mesh for 0 — 0. As expected, the eigen- 
value spectrum depends only weakly on the underlying mesh given that the mesh size 
is sufficiently small. Concerning the computational complexity of POD with nonnested 
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Figure 2.9: Run 2.3. Uniform mesh (left), disturbed meshes at t = 0.5 and t = 1.0 (middle left, middle 
right), and overlap of the mesh at t = 0 with the mesh at t = 1.0 (right). Here, we use 0 = 10. 
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Figure 2.10: Run 2.3: Decay of eigenvalues of matrix K with different meshes. 


meshes let us note that solving the heat equation takes 2.1 seconds on the disturbed 
meshes and 1.8 seconds on the uniform mesh. The computational time needed to com- 
pute each entry of the matrix K is 0.022 seconds and computing the eigenvalue decom- 
position for K takes 0.0056 seconds. Note that the cut element integration problem for 
each matrix entry takes a fraction of the time required to solve the finite element prob- 
lem. 


2.3 The POD-Galerkin procedure 


Once the POD basis is generated it can be used to set up a POD-Galerkin approxima- 
tion of the original dynamical system. This is discussed in the present section. In this 
context we recall that the space spanned by the POD basis is used with a Galerkin 
method to approximate the original system for, e. g., other inputs and/or parameters 
than those used to generate the snapshots for constructing the POD basis. A typical ap- 
plication is given by the PDE-constrained optimization, where the PDE system during 
the optimization is substituted by POD-Galerkin surrogates; see Section 2.6 for more 
details. 
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2.3.1 The POD-Galerkin procedure 


Suppose that for given snapshots y? € V” c X,1< j < n, we have computed the 
symmetric matrix 


K= (Vafa; WEY Y) isijen with rankK =r <n (2.30) 


associated to the operator K from (2.8) together with its eigensystem. Its £ € {1,...,r} 
largest eigenvalues are Aa with corresponding eigenvectors 1D c R”. The POD 
basis {¥;}{; is then given by (2.9), i. e., 


y,- -L yo, fori=1,...,2@. 
Ài 


VA; 


This POD basis is utilized in order to compute a reduced-order model for (2.16) 
along the lines of Section 2.2.3.3, where the space V” is replaced by the space V’ = 
span {V,,..., ¥,} c V. More precisely, we make the POD-Galerkin ansatz 


£ £ 
y(t) = YnlO¥; = Yn) yo, forallt € [0,7], (2.31) 
i=1 tal Vi; 


as an approximation for y(t), with the Fourier coefficients 


1 


VA; 


Inserting yf into (2.16) and choosing V° c V as the test space leads to the system 


ni(t) = y O, Vi) y = (ro vo), forl<i<é. 


d e e £ Z 
T (OP) +a H, E) + NYO) PY) yy = FO Y) a (2.32) 


(Y O), P)y = Yo Dy 


for all ¥ € Vf and for almost all t € (0, T]. The system (2.32) is called POD reduced- 
order model (POD-ROM). Using the ansatz (2.31), we can write (2.32) as an £-dimen- 
sional ordinary differential equation system for the POD mode coefficients y(t) = 
(ni(t))i<i<e; t € (0, T] as follows: 


4 r4 
YO PY) yt yO aH, Y) = FO -NV O) Yyy 
= is (2.33) 


£ 
YO) P Y) y = Yo Ya 
j=1 


fori = 1,...,2. Note that (¥;, Y;)y = 6, if we choose X = H in the context of Sec- 
tion 2.2.3. In a next step we rewrite this system using the relation between Y; and ®; 
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given in (2.9). This leads to 


e 
z VD; ) y 
Xi (t) VVP (YO; a Sn (t) a(yO, 2% FO-MNY HVP: )y y 
j=1 vii vai 
for t € (0, T], (2.34) 


e 
y n;(0) (VPY D) , — VVD)y 


jal JA yA: 


fori =1,..., £. In order to write (2.34) in a compact matrix-vector form, let us introduce 
the diagonal matrix 


. 1 1 exe 
A= diag ( eed Jer 
Vù Ve 


From the first £ eigenvectors Da of K we build the matrix 
= [D,|...| De] € R”, 
Then, the system (2.34) can be written as the system 


(ADTKPAJAE) + (AA AJNE) + AN'(n(0) = AF (E) fort e (0, k (2.35) 


A®'K®An(0) = An, 


for the vector-valued mapping n = (7,;...,4¢)" : [0,T] > Rf, for the nonlinearity 
= (NF())icice : R > Rf with 


NÉ (v) = (azan) o) = (azoo ho) ; 
v'v VV 


and for the stiffness matrix A‘ = ((Aj)) eR’? given as 
Aj =a(VO;,YO;) forlsijs<e. 


Note that the right-hand side F(t) = (F/(t)),<j<¢ and the initial condition n, = (.;)1<i<e 
are given by 


FiO = (FOV) yy = (VFO Pg» te [0,Tlae., 
and 
Noi = (Vos YD) H = (VoD) ass 


fori = 1,...,¢, respectively. Their calculation can be done explicitly for any arbitrary 
finite element discretization. For a given function w € V (for example w = f(t) or 
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w = y,) with finite element discretization w = xe W;Xi, nodal basis CAH c V, and 


appropriate mode coefficients {w; pa we can compute 


OW) = w= (Zwe shoh ) =È D u Gol 


forj = 1,...,n, where y? = ee vig, € V” denotes the j-th snapshot. Again, for any 
i = 1,...,m„ and k =1.,..., Mj, the computation of the inner product (y;, o) x can be 
done explicitly. 

Obviously, for linear evolution equations the POD-ROM (2.35) can be set up and 
solved using snapshots with arbitrary finite element discretizations. The computation 
of the nonlinear component N°(n(t)) needs particular attention. In Section 2.3.3 we 
discuss the options to treat the nonlinearity. 


2.3.2 Time-discrete reduced-order model 


In order to solve the reduced-order system (2.32) numerically, we apply the implicit 
Euler method for time discretization and use for simplicity the same temporal grid 
fth as for the snapshots. It is also possible to use a different time grid; cf. [58]. The 
time-discrete reduced-order model reads 


t 
Y -Yr Py a j F(t), Pyy 
D FRH ah) + NOP) yy = | — a ar, 

At; J J VV ; At; (2.36) 


sati 


(yi Y)y = Yo Y)y 


for all ¥ € Vf andj = 2,...,n. Equivalently the following system holds for the coeffi- 
cient vector y(t) € Rf (cf. (2.35)): 


i _ pil i ; 
(Ao*Koa)(2 ui ) + (AAA) + AN (f) = AFj, j=2,...,n, 
j (2.37) 
A®'K®An! = An, 
with the inhomogeneity Fj = (Fi icio] = . n, given as 


b bo 
F c Í (F(T), VD) v y d- | (Y*F (7), D;) an r 


fa fa 
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2.3.3 Discussion of the computation of the nonlinear term 


Let us now consider the computation of the nonlinear term AN’ (1) ) € Rf of the POD- 
ROM (2.35). We have 


(AN M) = (NOD) Yvy = Ciez mow Jre) 


V',V 


for k = 1,..., £. It is well known that the evaluation of nonlinearities in the reduced- 
order modeling context is computationally expensive. To make this clear, let us as- 
sume we are given a uniform finite element discretization with m degrees of freedom. 
Then, in the fully discrete setting, the nonlinear term has the form 


WWN"(Wn(t)) € Rf, te [0,T]a. e., 


where Y = [¥;,|...|‘%)] € IR” is the matrix in which the POD modes are stored 
columnwise and W € R™*" is a weighting matrix related to the utilized inner product 
(cf. (2.3)). Hence, the treatment of the nonlinearity requires the expansion of Yn(t) € 
R” in the full space for t € [0,T] a.e. Then the nonlinearity can be evaluated and 
finally the result is projected back to the POD space. Obviously, this means that the 
reduced-order model is not fully independent of the high-order dimension m and ef- 
ficient simulation cannot be guaranteed. Therefore, it is convenient to seek for hyper- 
reduction, i.e., for a treatment of the nonlinearity, where the model evaluation cost 
is related to the low dimension £. Common choices are empirical interpolation meth- 
ods like, e. g., the empirical interpolation method (EIM) [14], the discrete EIM (DEIM) 
[24], and the QR decomposition-based DEIM [31]. Another option is dynamic mode de- 
composition for nonlinear model order reduction; see, e. g., [7]. Furthermore, in [98] 
nonlinear model reduction is realized by replacing the nonlinear term by its inter- 
polation in the finite element space. Alternative approaches for the treatment of the 
nonlinearity are missing point estimation [10] and best points interpolation [70]. 

Most of these methods need a common reference mesh for the computations. To 
overcome this restriction we propose different paths which allow for more general dis- 
crete settings like r-adaptivity, discussed in Run 2.3. 

One option is to use EIM [14]. Alternatively, we can linearize and project the non- 
linearity onto the POD space. For this approach, let us consider the linear reduced- 
order system for a fixed given state y, which takes the form 


A Hey, Yyy + aO, Y) + NOO) Yyyy = FO, Y) ‘a 


dt (2.38) 
O), Yyy = Yo Yy 


for all ¥ € V° and for almost all t € (0, T]. The linear evolution problem (2.38) can be 
set up and solved explicitly without spatial interpolation. In the numerical examples 
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in Section 2.6, we take the finite element solution as given state in each time step, i.e., 
Y(t) = yj forj = 2,...,n. 

Furthermore, the linearization of the reduced-order model (2.32) can be consid- 
ered: 


“ VEO, Yy + OY) + (NOD. Yyy 
= (f()- NWO) +N PONIO: Yy v (2.39) 
(yO), Yyy = Y Yy 


for all ¥ € V° and for almost all t € (0, T], where N’ denotes the Fréchet derivative of 
the nonlinear operator M. This linearized problem is of interest, e. g., in the context 
of optimal control, where it occurs in each iteration level within sequential quadratic 
programming (SQP) methods; see [49], for example. Choosing the finite element solu- 
tion as given state in each time instance and using (2.9) leads to 


NY) Pid yey = a yVar(®; )k (NY), Ye) yr, y? 


VAi A 


(N'Y G) Fi) yey = (x'on( malty) ) 
£ 
= RO 5 5 ava u(P) (Opp NYY Vu) yry 


v=1 p=1 


MO» Pidy y = a > VaR PI) NOY pV) vy 


VV 


forj = 2,...,n andi = 1,...,¢@. Finally, we approximate the nonlinearity AN (rf! J)e R 
in (2.37) by 


(AN M); = VO) ENOO E -y Yyy 
forj =2,...,.nandi=1,...,€, which can be written as 


AN (f) = ADN + ADN DAY - AON, 


where 
NOY) VEY y y NM D)V pp VND) yr y 
N = ; eR", N, = l eR", 
(N)s Von) pry (N'Y) > Van) pry 
and with y; = a@jy;,j=1,...,n, 
MOI yy Nw Id yry 
N= : : Ta 


NY yy = NY e In yey 
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For weakly nonlinear systems this approximation may be sufficient, depending on the 
problem and its goal. A great advantage of linearizing the semi-linear partial differen- 
tial equation is that only linear equations need to be solved, which leads to a further 
speedup; see Table 2.6. However, if a more precise approximation is desired or nec- 
essary, we can think of approximations including higher-order terms, like quadratic 
approximation, see, e. g., [25] and [84], or Taylor expansions, see, e. g., [73, 74] and [35]. 
Nevertheless, the efficiency of higher-order approximations is limited due to growing 
memory and computational costs. 


2.3.4 Expressing the POD solution in the full spatial domain 


Having determined the solution (tf) to (2.35), we can set up the reduced solution y(t) 
in a continuous framework: 


y (t) = D nit) (Q;)jy; (2.40) 
2 ~ Vay; 


VA 


Now, let us turn to the fully discrete formulation of (2.40). For a time-discrete setting, 
we introduce for simplicity the same temporal grid {th as for the snapshots. The 
snapshots (2.28) admit the expansion 


Shits: 
y=} yo forj=1,...,n 
i1 


Let {Q Je denote an arbitrary set of grid points for the reduced system at time level t;. 
The fully discrete POD solution can be computed by evaluation: 


N= Yio 4 $ Vi (®; (3: Pr, »)) (2.41) 


Vii oa 


forr = 1,...,l,andj = 1,...,n. This allows us to use any grid for expressing the POD 
solution in the full spatial domain. For example, we can use the same nodes at time 
level j for the POD simulation as we have used for the snapshots, i.e., for j = 1,...,n 
we have l; =m; and Qi = P forallr,k =1,..., mj. Another option can be to choose 


{ai}, “UL {Pi} forj=1,...,n, 


i. e., the common finest grid. Obviously, a special and probably the easiest case con- 
cerning the implementation is to choose snapshots which are expressed with respect 
to the same finite element basis functions and utilize the common finest grid for the 
simulation of the reduced-order system, which is proposed by [94]. After expressing 
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the adaptively sampled snapshots with respect to a common finite element space, the 
subsequent steps coincide with the common approach of taking snapshots which are 
generated without adaptivity. Then, expression (2.41) simplifies to 


tj, P,) = Yno z È E ) forj =1,...,n, (2.42) 


v=1 


where {P,}"", are the nodes of the common finite element space. 


Run 2.4 ([39, Example 6.1]). Let us revisit Run 2.1 and consider its POD-Galerkin so- 
lutions. The POD solutions for £ = 10 and £ = 50 POD basis functions using spatial 
adaptive snapshots which are interpolated onto the finest mesh are shown in Fig- 
ure 2.11. As expected, the more POD basis functions we use (until stagnation of the 
corresponding eigenvalues), the fewer oscillations appear in the POD solution and the 
better the approximation is. Table 2.2 compares the approximation quality in the rel- 


Figure 2.11: Run 2.4. Surface plot of the POD solution using £ = 10 (top) and £ = 50 (bottom) POD 
basis functions at t = t (left), t = 7/2 (middle), and t = T (right). 


ative L?(0, T; L?(Q))-norm of the POD solution using adaptively generated snapshots 
which are interpolated onto the finest mesh with snapshots of uniform spatial dis- 
cretization depending on different POD basis lengths. Then, for £ = 20 we obtain a 
relative LO, T; L? (Q))- ~ between the POD solution and the finite element solution 
of size e24 = 3.08 - 10°? anda ae L?(0, T; L(Q))-error between the POD solution 
and the true solution of size eê% = 2.17- 107°. 

We note that et”! decays down to 1078 (£ = 100) and then stagnates if using a uni- 
form mesh. This behavior is clear, since the more POD basis elements we include (up to 
stagnation of the corresponding eigenvalues), the better an approximation A the POD 
solution for the finite element solution. On the other hand, both e*®i and e*4, start to 


true 


stagnate after £ = 30 in Table 2.2, columns 4 and 5. This is due to the fact that at this 
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Table 2.2: Run 2.4. Relative 17(0, T;L?(Q))-error between the POD solution and the finite element 
solution (columns 2 and 3) and the true solution (columns 4 and 5), respectively, using adaptive 
finite element snapshots which are interpolated onto the finest mesh and using a uniform mesh. 
t ere EE Ene etme 
1 1.30-10° 1.30-10° 1.28-10° 1.30-10° 
3 7.49.10! 7.58-107! 7.46-107! 7.60-107? 
5 439-107! 445-107! 439-101 446-107! 
10 137-107? 137-107? 1.36.10! 1.38-107 
20 3.08-107? 1.56-107? 2.17-10°? 1.60-10°? 
30 2.59-10°% 2.04-10°% 1.49-10°% 3.00.10? 
50 2.63-102 567-10 1.41-107? 2.07-10°3 
100 2.61-10°? 648-1078 1.40.10? 2.06.10 
150 2.61-10°? 813-1077 1.39.10? 2.07-1077 


Table 2.3: Run 2.4. Relative 17(0, T; H*(Q))-error between the POD solution and the finite element 
solution (columns 2 and 3) and the true solution (columns 4 and 5), respectively, using adaptive 
finite element snapshots which are interpolated onto the finest mesh and using a uniform mesh. 


ad uni ad uni 
£ EFE EFE Etrue Etrue 


1 1.46-10° 1.46-10° 1.46-10° 1.47-10° 

3 1.21-10° 1.22-10° =1.22-10° 1.22.10? 

5 939.107! 945-107! 9.47-101 9.51-107! 
10 422-107! 4.25.1071 4.33-107! 4.31-107! 
20 7.76-107? 7.27-107? 1.02-107! 8.19.10? 
30 2.92-10°% 1.22-10°% 7.26.1072? 3.52.10? 
50 2.61-107? 474-107 7.05-107? 3.27-10°? 
100 2.79-107? 4.78-107 6.94.10? 3.27-10°7 
150 2.93-10°7 2.84.1077 6.87-107? 3.27-10°? 


point the spatial (and temporal) discretization error dominates the modal error. This is 
in accordance with the decay of the eigenvalues shown in Figure 2.3 and is accounted 
for, e. g., in the error estimation presented in [39, Theorem 5.1]. Similar observations 
hold true for the relative L7(0, T; H'(Q))-error listed in Table 2.3 with the difference that 
the L?(0, T; H'(Q))-error is larger than the respective L7(0, T; L7(Q))-error. 

The computational times for the full and the low-order simulation using uniform 
finite element discretizations and adaptive finite element snapshots, which are inter- 
polated onto the finest mesh, respectively, are listed in Table 2.4. 

Once the POD basis is computed in the offline phase, the POD simulation corre- 
sponding to adaptive snapshots is 13,485 times faster than the finite element simu- 
lation using adaptive finite element meshes. This speedup factor is important when 
one considers, e. g., optimal control problems with time-dependent PDEs, where the 
POD-ROM can be used as surrogate model in repeated solution of the underlying PDE 
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Table 2.4: Run 2.4. CPU times for FE and POD simulation using uniform finite element meshes and 
adaptive finite element snapshots which are interpolated onto the finest mesh, respectively, and 
using £ = 50 POD modes. 


Adaptive FE mesh Uniform FE mesh Speedup factor 
FE simulation 944 sec 8,808 sec 9.3 


POD offline computations 264 sec 1,300 sec 4.9 
POD simulation 0.07 sec 0.07 sec = 
Speedup factor 13,485 125,828 - 


model. In the POD offline phase, the most expensive task is to express the snapshots 
with respect to the common finite element space, which takes 226 seconds. Since 
K (2.30) is symmetric, it suffices to calculate the entries on and above the diagonal, 
which are X% k = (n? +n)/2 entries. Thus, the computation of each entry in the corre- 
lation matrix K using a common finite element space takes around 0.00018 seconds. 
We note that in the approach explained in Sections 2.2.4 and 2.3, the computation of 
the matrix K is expensive. For each entry the calculation time is around 0.03 seconds, 
which leads to a computation time of around 36,997 seconds for matrix K. The same ef- 
fort is needed to build Af = a(ya@;, Y®;). In this case, the offline phase takes therefore 
around 88,271 seconds. For this reason, the approach to interpolate the adaptively 
generated snapshots onto the finest mesh is computationally more favorable. But 
since the computation of K can be parallelized, the offline computation time can be 
reduced provided that the appropriate hardware is available. 


Run 2.5 (Cahn-Hilliard equations). Now let us revisit Run 2.2, where in the following 
we run the numerical simulations for different combinations of numbers for £, and 
£„ of Table 2.1. The approximation quality of the POD solution using adaptive meshes 
is compared to the use of a uniform mesh in Table 2.5. As expected, Table 2.5 shows 
that the error between the POD surrogate solution and the high-fidelity solution gets 
smaller for an increasing number of utilized POD basis functions. Moreover, a larger 
number of POD modes is needed for the chemical potential w than for the phase field 
c in order to get an error in the same order, which is in accordance with the fact that 
the decay of the eigenvalues for w is slower than for c, as seen in Figure 2.8. 

We now discuss the treatment of the nonlinearity and also investigate the in- 
fluence of nonsmoothness of the model equations to the POD procedure. Using 
the convex-concave splitting for W, we obtain for the Moreau—Yosida relaxed dou- 
ble obstacle free energy the concave part W'(c) = 3a - c°) and the convex part 
wrl(c) = $(max(c - 1, 0)? + min(c + 1, 0)). This means that the first derivative of the 
concave part is linear with respect to the phase field variable c. The challenging part 
is the convex term with nonsmooth first derivative. For a comparison, we consider the 
smooth polynomial free energy with concave part W? (c) = ia — 2c’) and convex part 
WP) =i. 
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Table 2.5: Run 2.5. Relative 17(0, T; L?(Q))-error between the POD solution and the finite element 
solution using adaptive meshes (columns 3 and 4) and using a uniform mesh (columns 5 and 6), 
respectively. 


3 4 844-107 3.00-10° 844-10 3.75-10° 
10 13 3.30-:10 3.77-10' 3.30-10 432-107? 
19 26 1.57-10 212-107 157-103 2.39-107% 
29 26 7.34-10°* 1.09.10! 7.32-10°4 1.16-107? 
37 26 3.57-10% 482-107?  3.55-10°* 5.04-10°7 
50 50 188-107 2.17-107? 1.86-107% 2.33.10? 
65 26 9.74-10°° 1.11:-107? 9.56.10? 1.15-10°7 

100 100 3.37-107 3.56.10? 3.22-10°° 3.42.10? 


e W‘.(c) 
10° 10° 
polynomial polynomial 
relaxed double obstacle relaxed double obstacle 
10° 10° 
o o 
O QO 
È 101 EÈ 10% 
a < 
1075 105 
10 10 
0 200 400 600 800 1000 0 200 400 600 800 1000 
index i index i 


Figure 2.12: Run 2.5. Comparison of the normalized eigenvalues for c (left) and the first derivative of 
the convex part W} of the free energy (right) using polynomial and relaxed double obstacle energy, 
respectively. 


Figure 2.12 shows the decay of the normalized eigenspectrum for the phase field c (left) 
and the first derivative of the convex part W’ (c) (right) for the polynomial and the re- 
laxed double obstacle free energy. Obviously, in the nonsmooth case more POD modes 
are needed for a good approximation than in the smooth case. This behavior is similar 
to the decay of the Fourier coefficients in the context of trigonometric approximation, 
where the decay of the Fourier coefficients depends on the smoothness of the approx- 
imated object. 

Table 2.6 summarizes computational times for different finite element runs as well 
as reduced-order simulations using the polynomial and the relaxed double obsta- 
cle free energy, respectively. In addition, the approximation quality is compared. The 
computational times are rounded averages from various test runs. It turns out that 
the finite element simulation (row 1) using the smooth potential is around two times 
faster than using the nonsmooth potential. This is due to the fact that in the smooth 
case, two to three Newton steps are needed for convergence in each time step, whereas 
in the nonsmooth case six to eight iterations are needed in the semi-smooth Newton 
method. 
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Table 2.6: Run 2.5. Computational times, speedup factors, and approximation quality for different 
POD basis lengths and using different free energy potentials. 


wP w: 

FE 1644s 3,129s 

€,=3 €, = 19 €,=3 €, = 19 

fy =4 ey = 26 fy =4 ly = 26 
POD offline 355s 355s 350s 349s 
DEIM offline 8s 8s 9s 10s 
ROM 183s 191s 2,6165 3,3885 
ROM-DEIM 0.05s 0.15 0.04s no conv. 
ROM-proj 0.008 s 0.03s 0.01s 0.03s 
speedup FE-ROM 8.9 8.6 1.1 none 
speedup FE-ROM-DEIM 32,880 16,440 78,225 - 
speedup FE-ROM-proj 205,500 54,800 312,900 104,300 
rel L7(Q) error ROM 5.46-10° 3.23-107 844-10 1.57-10° 


rel L?(Q) error ROM-DEIM 1.46-10°2.  3.83-10°* 8.84.10 = 
rel L7(Q) error ROM-proj 470-10? 418-10? 872-107 9.80-10°3 


Using the smooth polynomial free energy, the reduced-order simulation is eight to nine 
times faster than the finite element simulation, whereas using the relaxed double ob- 
stacle free energy it only delivers a very small speedup. The inclusion of DEIM (we 
use fdeim = €,) in the reduced-order model leads to immense speedup factors for both 
free energy functions (row 8). This is due to the fact that the evaluation of the non- 
linearity in the reduced-order model is still dependent on the full spatial dimension 
and hyperreduction methods are necessary for useful speedup factors. Note that the 
speedup factors are of particular interest in the context of optimal control problems. 
At the same time, the relative L?(0, T; L*(Q))-error between the finite element solution 
and the ROM-DEIM solution is close to the quality of the reduced-order model solution 
(rows 10 and 11). 

However, in the case of the nonsmooth free energy function using £. = 19 POD 
modes for the phase field and £, = 26 POD modes for the chemical potential, the 
inclusion of DEIM has the effect that the semi-smooth Newton method does not con- 
verge. For this reason, we treat the nonlinearity by applying the technique explained 
in Section 2.3.1, i. e., we project the finite element snapshots for W! (c) (which are inter- 
polated onto the finest mesh) onto the POD space. Since this leads to linear systems, 
the computational times are very small (row 6). The error between the finite element 
solution and the reduced-order solution using projection of the nonlinearity is of the 
magnitude 107?/107°. Depending on the motivation, this approximation quality might 
be sufficient. Nevertheless, we note that for large numbers of POD modes, using the 
projection of the nonlinearity onto the POD space leads to a large increase of the error. 
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To summarize, a POD-ROM construction approach is proposed which can be set 
up and solved for snapshots originating from arbitrary finite element (and also other) 
spaces. The method is applicable for h-, p-, and r-adaptive finite elements. It is mo- 
tivated from an infinite-dimensional perspective. Using the method of snapshots we 
are able to set up the correlation matrix K from (2.30) by evaluating the inner prod- 
ucts of snapshots which live in different finite element spaces. For nonnested meshes, 
this requires the detection of cell collision and integration over cut finite elements. A 
numerical strategy how to implement this practically is elaborated and numerically 
tested. Using the eigenvalues and eigenvectors of this correlation matrix, we are able 
to set up and solve a POD surrogate model that does not need the expression of the 
snapshots with respect to the basis of acommon finite element space or the interpola- 
tion onto a common reference mesh. Moreover, an error bound for the error between 
the true solution and the solution to the POD-ROM using spatially adapted snapshots 
is available in [39, Theorem 5.1]. The numerical tests show that the POD projection er- 
ror decreases if the number of utilized POD basis functions is increased. However, the 
error between the POD solution and the true solution stagnates when the spatial dis- 
cretization error dominates. Moreover, the numerics show that using the correlation 
matrix calculated explicitly without interpolation in order to build a POD-ROM gives 
the same results as the approach where the snapshots are interpolated onto the finest 
mesh. From a computational point of view, sufficient hardware should be available 
in order to compute the correlation matrix in parallel and make the offline computa- 
tional time competitive. For semi-linear evolution problems, the nonlinearity is treated 
by linearization. This is of interest in view of optimal control problems, in which a 
linearized state equation has to be solved in each SQP iteration level. An appropri- 
ate treatment of the nonlinearity in our applications gains significant speedup of the 
reduced-order model with respect to computational time when compared to the full 
simulations. This makes POD-ROM with adaptive finite elements an ideal approach 
for the construction of surrogate models in, e. g., optimal control with nonlinear PDE 
systems, as they arise, e. g., in the context of multiphase flow control problems. 


2.4 Certification with a priori and a posteriori error 
estimates 


As we have seen in Section 2.3, POD provides a method for deriving low-order mod- 
els of dynamical systems. It can be thought of as a Galerkin approximation in the 
spatial variable, built from functions corresponding to the solution of the physical 
system at prespecified time instances. After carrying out SVD, the leading £ gener- 
alized eigenfunctions are chosen as the POD basis EA of rank £. As soon as one 
uses POD, questions concerning the quality of the approximation properties, conver- 
gence, and rate of convergence become relevant. Let us refer, e.g., to the literature 
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[22, 42, 56, 58, 57, 85, 88, 89, 80] for a priori error analyses of POD-Galerkin approxi- 
mations. It turns out that the error depends on the decay of the sum )’;,, A;, the error 
At’ (with an appropriate B > 1) due to the used time integration method, the used 
Galerkin spaces {vii ets and the choice X = H or X = V. In particular, best approx- 
imation properties hold provided the time differences y(t) (or the finite difference 
discretizations) are included in the snapshot ensembles; cf. [56, 58, 89]. 

Let us recall numerical test examples from [42, Section 1.5]. The programs are writ- 
ten in MATLAB using the PARTIAL DIFFERENTIAL EQUATION TOOLBOX for the computa- 
tion of the piecewise linear finite element discretization. For the temporal integration 
the implicit Euler method is applied based on the equidistant time grid t; = (j - 1)At, 
j=1,...,n, and At = T/(n-1). 


Run 2.6 (POD for the heat equation; cf. [42, Run 1]). We choose the final time T = 3, 
the spatial domain Q = (0, 2) c R, the Hilbert spaces H = L?(Q), V= H5(Q), the source 
term f(t,x) = P — x? for (t,x) € Q = (0,T) x Q, and the discontinuous initial value 
Y. (X) = X(0.5,1.0) -Xa1.5) for x € Q, where, e. g., X(9.5,1) denotes the characteristic function 
on the subdomain (0.5, 1) € Q, X(9.5,1)(x) = 1forx € (0.5, 1) and X(o51)(x) = 0 otherwise. 
We consider a discretization of the linear heat equation (compare (2.17) with c = 0) 


y(t, x) — Ay(t,x) =f(t,x) for (t,x) €Q, 
y(t,x) =0 for (t,x) € È = (0, T) x aQ, (2.43) 
y(0,x) =y (x) forxeQ. 


To obtain an accurate approximation of the exact solution we choose n = 4,000 so 
that At = 7.5 - 107“ holds. For the finite element discretization we choose m = 500 
spatial grid points and the equidistant mesh size h = 2/(m + 1) = 4 - 107°. Thus, the 
finite element error — measured in the H-norm -is of the order 10~™*. In the left graphic 
of Figure 2.13, the finite element solution y” to the state equation (2.43) is visualized. 
To compute a POD basis Pa of rank £ we utilize the multiple discrete snapshots 
yj = y"(t) for 1 < j < n, as well y? = 0 and y = (y(t) - y(t, 4))/At j E2 Migs ds Os 
we include the temporal difference quotients in the snapshot ensemble and K = 2, 
n; =n, = n,. We choose X = H and utilize the (stable) SVD to determine the POD basis 
of rank £; compare Section 2.2.2. We address this issue in a more detail in Run 2.9. 
Since the snapshots are finite element functions, the POD basis elements are also finite 
element functions. In the right plot of Figure 2.13, the projection and reduced-order 
error given by 
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Figure 2.13: Run 2.6 (cf. [42, Figure 1.1]). The FE solution y” (left) and the residuals corresponding to 
the POD basis rank £ (right). 


are plotted for different POD basis ranks £. The chosen trapezoidal weights a; have 
been introduced in (2.23). We observe that both errors decay rapidly and coincide until 
the accuracy 10°”, which is already significantly smaller than the finite element dis- 
cretization error. These numerical results reflect the a priori error estimates presented 
in [42, Theorem 1.29]. 


Run 2.7 (POD for a convection dominated heat equation; cf. [42, Run 2]). Now we con- 
sider a more challenging example. We study a convection-reaction-diffusion equation 
with a source term which is close to being singular: Let T, Q, y,, H, and V be given as 
in Run 2.6. The parabolic problem reads as follows: 


yilt, X) — CVxx(t, x) + BY,(6X) + ay(t, x) = f(6x) for (t,x) € Q, 
y(t,x) =0 for (t,x) € È, 
y(0, x) = y.(x) forx € Q. 


We choose the diffusivity c = 0.025, the velocity 8 = 1.0 that determines the speed in 
which the initial profile y, is shifted to the boundary, and the reaction rate a = —0.001. 
Finally, f(t,x) = P(4) cos(nx) for (t,x) € Q, where (Pz)(t) = min(+/, max(-l, z(t))) 
restricts the image of z on a bounded interval. In this situation, the state solution y 
develops a jump at t = 1 for! — oo; see the left plot of Figure 2.14. The right plot 
of Figure 2.14 demonstrates that in this case, the decay of the reconstruction residu- 
als and the decay of the errors are much slower than in the right plot of Figure 2.13. 
The manifold dynamics of the state solution require an inconveniently large number 
of POD basis elements. Since the supports of these ansatz functions in general cover 
the whole domain Q, the corresponding system matrices of the reduced model are not 
sparse. This is different for the matrices arising in the finite element Galerkin frame- 
work. Model order reduction is not effective for this example if a good accuracy of the 
solution function y’ is required. Strategies to improve the accuracy and robustness of 
the POD-ROM in those situations are discussed in, e. g., [18, 99] 
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Figure 2.14: Run 2.7 (cf. [42, Figure 1.2]). The FE solution y” (left) and the residuals corresponding to 
the POD basis rank £ (right). 


Run 2.8 (True and exact approximation error; cf. [42, Run 3]). We consider the setting 
introduced in Run 2.6 again. The exact solution to (2.43) does not possess a represen- 
tation by elementary functions. Hence, the presented reconstruction and reduction 
errors actually are the residuals with respect to a high-order finite element solution y”. 
To compute an approximation y of the exact solution y,, we apply a Crank-Nicolson 
method (with Rannacher smoothing [77]) ensuring |ly — Yexllz2(0,7.4) = O(At? +h?) = 
10%. In the context of model reduction, such a state is sometimes called the “true” 
solution. To compute the finite element state y" we apply the Euler method. In the left 
plot of Figure 2.15 we compare the true solution y,, with the associated POD approxi- 
mation for different values n, € {64, 128, 256, ..., 8192} of the time integration and for 
the spatial mesh size h = 4- 107°. For the norm we apply a discrete L7(0, T; H)-norm as 
in Run 2.6. Let us mention that we compute for every n, a corresponding finite element 
solution y”. We observe that the residuals ignore the errors arising by the application 
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Figure 2.15: Run 2.8 (cf. [42, Figure 1.3]). The reduced-order model errors with respect to the true 
solution (left) and the exact one (right). 
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of time and space discretization schemes for the full-order model. The errors decay be- 
low the discretization error 10~°. If these discretization errors are taken into account, 
the residuals stagnate at the level of the full-order model accuracy instead of decay- 
ing to zero; cf. the right plot of Figure 2.15. Due to the implicit Euler method we have 
lly" - Yexllz2¢0,rw = O(At + h°) with the mesh size h = 4 - 10%. In particular, from 
n, € {64, 128, 256, ..., 8192} it follows that At > 3 - 107% > k? = 1.6 - 10™. Therefore, the 
spatial error is dominated by the time error for all values of n,. We can observe that 
the exact residuals do not decay below a limit of the order At. One can observe that for 
fixed POD basis rank £, the residuals with respect to the true solution increase if the 
high-order accuracy is improved by enlarging n,, since the reduced-order model has 
to approximate a more complex system in this case, where the residuals with respect 
to the exact solution decrease due to the lower limit of stagnation At = 3/(n; — 1). 


Run 2.9 (Different strategies for a POD basis computation; cf. [42, Run 4]). As ex- 
plained in Section 2.2.2, let Y € IR” denote the matrix of snapshots with rank r, let 
W e R™™ be the (sparse) spatial weighting matrix consisting of the elements (Qj Pidx 
(introduced in Section 2.2.3.3), and let D € R®” be the diagonal matrix containing the 
nonnegative weighting parameters ay . As explained in Section 2.2.2, the POD basis 
CARR of rank £ < r can be determined by providing an eigenvalue decomposition of 
the matrix YY" = W?YDYTW!? e R™™ one of Y'Y = D'Y? YTWYD"? e R™", or an 
SVD of Ë = W!?yp"? e R™®", Since n > min Runs 2.6-2.8, the first variant is the 
cheapest one from a computational point of view. In case of multiple space dimensions 
or if a second-order time integration scheme such as some Crank—Nicolson technique 
is applied, the situation is converse. On the other hand, an SVD is more accurate 
and stable than an eigenvalue decomposition if the POD elements corresponding to 
eigenvalues/singular values which are close to zero are taken into account: Since 
A; = o? holds for all eigenvalues A; and singular values g;, the singular values are able 
to decay to machine precision, where the eigenvalues stagnate significantly above. 
This is illustrated in the left graphic of Figure 2.16. Indeed, for £ > 20 the EIG-ROM 
system matrices become singular due to the numerical errors in the eigenfunctions 
and the reduced-order system is ill-posed in this case, while the SVD-ROM model 
remains stable. In the right plot of Figure 2.16 POD elements are constructed with 
respect to different scalar products and the resulting reduced-order model errors are 
compared: || - ||7-residuals for X = H (denoted by POD(H)), || - |ly-residuals for X = V 
(denoted by POD(V)), and || - ||y-residuals for X = H (denoted by POD(H,V)), which 
also works quite well, the consideration of time derivatives in the snapshot sample 
(denoted by POD(H,dt)), which allows to apply the a priori error estimate given in 
[42, Theorem 1.29-2)], and the corresponding sums of singular values (denoted by 
SV(H,dt)) corresponding to the unused eigenfunctions in the latter case which indeed 
nearly coincide with the reduced-order model errors. 
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Figure 2.16: Run 2.9 (cf. [42, Figure 1.4]). Singular values g; using the SVD (SVD Vals) or the eigen- 
value decomposition (EIG Vals) and the associated reduced-order model errors (SVD Error and EIG 
Error, respectively) (left); reduced-order model errors for different choices for X, the error norm, and 
the snapshot ensembles (right). 


Note that in many applications, the quality of the reduced-order model does not 
vary significantly if the weights matrix W refers to the space X = H or X = V andif 
time derivatives of the used snapshots are taken into account or not. Especially, the 
reduced-order model residual decays with the same order as the sum over the remain- 
ing singular values, independent of the chosen geometrical framework. 


2.5 Optimal snapshot location for computing POD 
basis functions 


The construction of reduced-order models for nonlinear dynamical systems using POD 
is based on the information carried of the so-called snapshots. These provide the spa- 
tial distribution of the nonlinear system at discrete time instances. Thus, we are inter- 
ested in optimizing the choice of these time instances in such a manner that the error 
between the POD solution and the trajectory of the dynamical system is minimized. 
This approach was suggested in [59] and was extended in [64] to parameterized ellip- 
tic problems. Let us briefly mention some related issues of interest. In [26, 32] the sit- 
uation of missing snapshot data is investigated and gappy POD is introduced for their 
reconstruction. An important alternative to POD model reduction is given by reduced 
basis approximations; we refer to [72] and references given there. In [37] a reduced 
model is constructed for a parameter-dependent family of large-scale problems by an 
iterative procedure that adds new basis variables on the basis of a greedy algorithm. 
In the PhD thesis [20] a model reduction is sought of a class for a family of models 
corresponding to different operating stages. 
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Suppose that we are given the n, snapshots DEDE c V c X. The goal is to de- 
termine additional k snapshots at time instances T = (Tj,...,T,) with O < t <T, 


j=1,...,k. In [59] we propose to determine T = (T4, .. . , Tk) by solving the optimization 
problem 
T 
7 e 2 
— ; 2. 
eee | lv) -y" Olly dt (2.44) 
0 


where y and yf are the solutions to (2.16) and its POD-Galerkin approximation, respec- 
tively. Clearly, the definition of the operator R given in (2.6) has to be modified as fol- 
lows: 


nh k 
RW => aj (y(t), ¥) WG) + È anaj (V(t), ¥)y VT) 
jel jel 


with appropriately modified (trapezoidal) weights aj, j=1,...,k+n,. Consequently, 
(2.44) becomes an optimization problem subject to the equality constraints 


RW, =AW, i=1,...,€. 


Note that no precautions are made in (2.44) to avoid multiple appearance of a snap- 
shot. In fact, this would simply imply that a specific snapshot location should be given 
a higher weight than others. While the presented approach shows how to choose op- 
timal snapshots in evolution equations, a similar strategy is applicable in the context 
of parameter-dependent systems. 

It turns out in our numerical tests carried out in [59] that the proposed criterion is 
sensitive with respect to the choice of the time instances. Moreover, the tests demon- 
strate the feasibility of the method in determining optimal snapshot locations for con- 
crete diffusion equations. 


Run 2.10 (cf. [59, Run 1]). For T = 1 let Q = (0, T) x Q and Q = (0,1) x (0,1) c R°. For 
the finite element triangulation we choose a uniform grid with mesh size h = 1/40, 
i. e., we have 900 degrees of freedom for the spatial discretization. Then, we consider 


yilt, x) — cAy(t,x) +B Vylt, x) + y(t, x) =fœ) for (t,x) € Q, 
c Ya, x)+q0)y(t,x)=g(x) for (t,x) €Z, 
y(0,x)=y.(x)  forx <Q, 
where c = 0.1, B = (0.1,-10)" € R, 


4 forall x = (x4, X) with (x, — 0.25)? + (x) - 0.65)? < 0.05, 
O otherwise, 


foe = | 
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Figure 2.17: Run 2.10 (cf. [59, Figures 3 and 4]). Initial condition y, (left plot) and FE solution y’ for 
t = 0.3 (middle) and t = T (right plot). 


and y, (X) = sin(7x,) cos(7x>) for x = (X1,X2) € Q (Figure 2.17, left plot). Furthermore, 
we have 


1 forx = (x%,1) withO <x <1, 
2 forx = (1,x) withO < x < 1, 
-2 forx = (xı, 0) with O < x, < 1, 
O forx = (0,x)withO < x <1, 


q(x) = 


1 for x = (x, 1)withO <x <1, 
g(x) = O forx = (1,xz) with O < x, < 1, for x = (0,x,) with O < x) <1, 
-1 forx = (x%,,0) withO <x, <1. 


We utilize piecewise linear finite element functions. The finite element solutions y" = 
y(t, x) for t = 0.15 and t = T are shown in Figure 2.17. Next we take snapshots on the 
fixed uniform time grid t; = G - 1)At, 1 < j < m, with n; = 10 and At = T/n, = 0.1. 
The goal is to determine four additional time instances t = (t,,...,t,) € [0, T] based on 
a finite element approximation for (2.44). Since the behavior of the solution exhibits 
more change during the initial time interval [0,0.3] than later on, we initialize our 
quasi-Newton method by the starting value T° = (0.05, 0.15, 0.25, 0.35) € [0,T]. The 
number of POD ansatz functions is fixed to be £ = 3. The corresponding value of the 
reduced-order model error is approximately 0.1093. The optimal solution is given as 
T = (0.0092, 0.0076, 0.1336, 0.2882) € [0, T], while the associated reduced-order model 
error is approximately 0.0165, which is a reduction of about 85 %. In Figure 2.18 we can 
see that the shapes of the three POD bases change significantly from the initial time 
instances T° € Rf to the optimal ones 7 € R4. 


2.6 Optimal control with POD surrogate models 


Reduced-order models are used in PDE-constrained optimization in various ways; see, 
e.g., [50, 86] for a survey. In optimal control problems it is sometimes necessary to 
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Figure 2.18: Run 2.10 (cf. [59, Figures 5 and 7]). POD basis Y4, Y3, Y, for the initial additional time 
instances t? € R4 (upper three plots) and for the optimal additional time instances T € R (lower 
three plots). 


compute a feedback control law instead of a fixed optimal control. In the implementa- 
tion of these feedback laws models of reduced-order can play an important and very 
useful role; see [11, 40, 60, 65, 68, 79]. Another useful application is the use in opti- 
mization problems, where a PDE solver is part of the function evaluation. Obviously, 
thinking of a gradient evaluation or even a step size rule in the optimization algorithm, 
an expensive function evaluation leads to an enormous amount of computing time. 
Here, the reduced-order model can replace the system given by a PDE in the objective 
function. It is quite common that a PDE can be replaced by a five- or ten-dimensional 
system of ordinary differential equations. This results computationally in a very fast 
method for optimization compared to the effort for the computation of a single solu- 
tion of a PDE. There is a large amount of literature in engineering applications in this 
regard; we mention only the papers [67, 71]. Recent applications can also be found in 
finance using the reduced models generated with the reduced basis method [76] and 
the POD model [85, 88] in the context of calibration for models in option pricing. 

We refer to the survey article [42], where a linear quadratic optimal control prob- 
lem in an abstract setting is considered. Error estimates for the POD-Galerkin approx- 
imations of the optimal control are proved. This is achieved by combining techniques 
from [28, 29, 44] and [56, 58]. For nonlinear problems we refer the reader to [50, 75, 86]. 
However, unless the snapshots are generating a sufficiently rich state space or are com- 
puted from the exact (unknown) optimal controls, it is not clear a priori how far the 
optimal solution of the POD problem is from the exact one. On the other hand, the POD 
method is a universal tool that is applicable also to problems with time-dependent 
coefficients or to nonlinear equations. Moreover, by generating snapshots from the 
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real (large) model, a space is constructed that inhibits the main and relevant physical 
properties of the state system. This, and its ease of use, makes POD very competitive 
in practical use, despite a certain heuristic flavor. In this context results fora POD a 
posteriori analysis are important; see, e. g., [93] and [41, 54, 55, 91, 92, 95, 97]. Using 
a fairly standard perturbation method it is deduced how far the suboptimal control, 
computed on the basis of the POD model, is from the (unknown) exact one. This idea 
turned out to be very efficient in our examples. It is able to compensate for the lack of 
a priori analysis for POD methods. Let us also refer to the papers [30, 36, 69], where 
a posteriori error bounds are computed for linear quadratic optimal control problems 
approximated by the reduced basis method. 

Data- and/or simulation-based POD models depend on the data (e. g., initial val- 
ues, right-hand sides, boundary conditions, observations, etc.) which are used to gen- 
erate the snapshots. If those models are used as surrogates in, e. g., optimization prob- 
lems with PDE constraints, the algorithmical framework has to account for this fact by 
providing mechanisms for accordingly updating the surrogate model during the solu- 
tion process. Strategies proposed in this context for optimal flow control can be found 
in, e. g., [3, 4, 9, 34, 17]. One of the most mature methods developed in this context is 
trust-region POD, proposed in [9], which since then has successfully been applied in 
many applications. We also refer to the work [38], where strategies for updating the 
POD bases are compared. 

The quality of the surrogate model highly depends on its information basis, which 
for snapshot-based methods is given by the snapshot set; compare Section 2.5. The 
location of snapshots and also the choice of the initial control in surrogate-based op- 
timal control are discussed in [5]. There, techniques from time-adaptive schemes for 
optimality systems of parabolic optimal control problems are adjusted to compute op- 
timal time locations for snapshots generation in POD surrogate modeling for parabolic 
optimal control problems. 

Concepts for the construction and use of POD surrogate modeling in robust op- 
timal control of electrical machines are presented in [63, 6]. Those problems are gov- 
erned by nonlinear partial differential equations with uncertain parameters, so that 
robustness can be achieved by considering a worst case formulation. The resulting 
optimization problem then is of bilevel structure and POD-ROMs in combination with 
a posteriori error estimators are used to speed up the numerical computations. 


2.7 Miscellaneous 


POD model order reduction (POD-MOR) can also be applied to provide surrogate mod- 
els for high-fidelity components in networks. The general perspective is discussed in, 
e. g., [48]. Related research for model order reduction of electrical networks is reported 
in, e. g., [16, 46, 47]. The basic idea here consists in a decoupling of MOR approaches 
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for the network and high-fidelity components which in general are modeled by PDE 
systems. For the latter, simulation-based POD-MOR techniques are used to construct 
surrogate models which then are stamped back into the (reduced) electrical network. 
Details and performance tests are reported, e. g., in [45, 47]. A short lecture series with 
related topics is presented under Hinze-Pilsen.” Further contributions to this topic can 
be found in [15]. 

Recent trends in data-driven and nonlinear MOR methods are discussed within a 
YouTube lecture series under Carlberg-YouTube.? 
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Francisco Chinesta and Pierre Ladevéze 
3 Proper generalized decomposition 


Abstract: The so-called “reduced” models have always been very popular and often es- 
sential in engineering to analyze the behavior of structures and materials, especially 
in dynamics. They highlight the relevant information and lead, moreover, to less ex- 
pensive and more robust calculations. In addition to conventional reduction methods, 
a generation of reduction strategies is now being developed, such as proper general- 
ized decomposition (PGD), which is the subject of this chapter. The primary feature of 
these strategies is to be very general and to offer enormous potential for solving prob- 
lems beyond the reach of industrial computing codes. It is typically the case when 
trying to take into account the uncertainties or the variations of parameters or non- 
linear problems with very large number of degrees of freedom, in the presence of sev- 
eral scales or interactions between several physics. These methods, along with the 
notions of “offline” and “online” calculations, also open the way to new approaches 
where simulation and analysis can be carried out almost in real-time. What distin- 
guishes PGD from proper orthogonal decomposition (POD) and reduced basis is the 
calculation procedure that does not differentiate between the different variables pa- 
rameters/time/space. In other terms, we can say that we minimize or make stationary 
a residual defined over the parameters-time-space domain. PGD with time/space sep- 
aration and the classical greedy computation technique were introduced in the 1980s 
as part of the LATIN solver [66, 67] for solving nonlinear time-dependent problems 
with the terminology “time/space radial approximation.” The corpus of literature de- 
voted to this method is vast [68, 77] but remained in the form of time/space separations 
for many years. A more general separated representation was more recently employed 
in [5, 6] for approximating the solution of multidimensional partial differential equa- 
tions. In [93], such separated representations are also considered for solving stochastic 
equations. PGD is the common name coined in 2010 by the authors of this chapter for 
these techniques because it can be viewed as an extension of the classical POD. To- 
day, many works use and develop the PGD in extremely varied fields. In this chapter we 
revisit the fundamentals, variants, and applications of PGD, covering different kinds 
of separated representations of the involved unknown fields as well as different con- 
structors able to address a variety of linear and nonlinear models, elliptic, parabolic, 
and hyperbolic. 
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In this chapter we use different notations to be consistent with the referred publica- 
tions. In any case, notation will be appropriately defined before being used to avoid any 
possible confusion. 


Keywords: MOR, PGD, separated representations, real-time, parametric models, 
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3.1 PGD: fundamentals 


3.1.1 Principles 


The proper generalized decomposition (PGD) method belongs together with reduced 
basis and proper orthogonal decomposition (POD) methods to the last generation of 
reduced-order model (ROM) computational methods characterized by a very large 
scope of applications. They are all based on the same concepts. The first and main 
idea in these methods is that the shape functions are not a priori given as usual. They 
are computed simultaneously with the solution itself thanks to an iterative proce- 
dure. For a problem defined over a parameter-time-space domain, the solution is then 
written as 


m 
Smt, X) = X$ aY, t, X) over X, x [0, T] x Q, (3.1) 
i=1 
where Y; are the normalized shape functions. 
The second idea is to introduce a variable separation hypothesis or something 
equivalent: 


Fiq t, X) = yi(WAj(t)A,(X), (3.2) 


where the scalar time functions A;, the scalar parameter functions y;, and the space 
functions A; are arbitrary. That is a deflection from other approximation methods for 
which shape functions are a priori given or partially given. Very often, a low-rank 
canonical format approximation is used. 

PGD is characterized by a global residual defined over the parameter-time-space 
domain which should be minimized. Let R be this residual, given as 


T 


R(s) = J | | du... du„dtdQ r(s) (3.3) 


with 


r20; r=0 over ÈX, x [0,T]xQ = S= Sexact: (3.4) 
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The problem to solve is then 


minr È yian- vrao } (3.5) 


i=1 


which is twice nonlinear. From a mechanical point of view, it could be nonlinear. Fur- 
thermore, the computation of a PGD approximation is always a nonlinear problem. 
To solve this minimization problem, the common technique is a “greedy” one. Such a 
technique has been introduced for PGD at the end of the 1980s [68]. It is an iterative 
technique in which, at iteration m + 1, the correction As = Sm+1 — Sm minimizes the 
residual: 


As = y'(uy).--¥"UnA(OMX): min R(Sm + As). (3.6) 
PATEA at, A 


The residual is minimized alternatively over the scalar parameter functions, the scalar 
time function, and the space function. Practically, few iteration loops are performed to 
get the new PGD mode. It has also been numerically observed that better performance 
could be obtained by updating the time and parameter functions of the previous PGD 
modes before computing a new PGD mode by minimizing the global residual. 

The classical PGD format has a tensorial equivalent: the canonical polyadic (CAN- 
DECOMP/PARAFAC, CP) format. The greedy procedure used in the PGD method is very 
close to tensorial tools such as the greedy CP alternating least-squares (ALS) algorithm 
[61], one major difference being the norm used: This algorithm minimizes the Frobe- 
nius norm of the error, while any norm, such as an energetic one, can be used in the 
PGD procedure. Other classical tensor formats can be used. The Tucker format, com- 
puted from an initial full tensor through higher-order singular value decomposition 
(SVD), is a compressed representation which gives a tensorial equivalent of the matrix 
concept of SVD. More recent structured representations have been developed such as 
tensor train or the hierarchical Tucker format. The former is a generic and simple for- 
mat which can have a better compression ratio than the CP decomposition. The latter 
is the more general structured format and includes all the previously presented ones. 
Its use can be delicate as the potentially complex structure must be chosen before any 
computation. These two formats have both been adapted to the resolution of mechan- 
ical partial differential equations in [99]. 


Remark. The simplest case is the situation where one computes the PGD of a given 
function defined over the time-space domain. It was proved in this case that conver- 
gence properties could be obtained and that the time functions are the eigenfunctions 
of a certain eigenvalue problem [68]. For the L?-norm and the discretized problem, the 
so-called PGD corresponds exactly to the classical SVD. It follows that PGD modes can 
be seen as “eigenmodes” and that the PGD can be seen as an extension of the SVD to 
partial differential equations. 
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3.1.2 Different types of separated representations 


Most of the existing model reduction techniques proceed by extracting a suitable re- 
duced basis and then projecting on it the problem solution. Thus, the reduced ba- 
sis construction precedes its use in the solution procedure, and one must be careful 
on the suitability of a particular reduced basis when employed for representing the 
solution of a particular problem. This issue disappears if the approximation basis is 
constructed at the same time as the problem is solved. Thus, each problem has its as- 
sociated basis in which its solution is expressed. One could consider few terms in its 
approximation, leading to a reduced representation, or all the terms needed for ap- 
proximating the solution up to a certain accuracy level. 

When calculating the transient solution of a generic problem u(x, t) we usually 
consider a given basis of space functions N,(x), i = 1,...,N,, the so-called shape func- 
tions within the finite element framework, and approximate the problem solution as 


Nn 
u(x, t) = X aN, (3.7) 


i=1 


which implies a space-time separated representation where the time-dependent co- 
efficients a;(t) are unknown at each time (when proceeding incrementally) and the 
space functions N;(x) are given “a priori,” e. g., by means of a polynomial basis. 

POD and reduced basis methodologies consider a reduced basis ,(x) for approx- 
imating the solution instead of using the generic functions N,(x). The former are ex- 
pected to be more suitable for approximating the problem at hand. Thus, it results 
that 


R 
u(x,t) = X bip, (3.8) 


i=1 


where in general R « N,,. Again (3.8) represents a space-time separated representa- 
tion where the time-dependent coefficient must be calculated at each time during the 
incremental solution procedure. 

Inspired by these results one could consider the general space-time separated rep- 
resentation 


N 
u(x,t) = X X;(x)T\(t), (3.9) 


i=1 


where now neither the time-dependent functions T;(t) nor the space functions X;(x) 
are “a priori” known. Both will be computed on-the-fly when solving the problem. 
As soon as one postulates that the solution of a transient problem can be ex- 
pressed in the separated form (3.9) whose approximation functions X;(x) and T;(t) will 
be determined during the problem solution, one could make a step forward and as- 
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sume that the solution of a multidimensional problem u(x,,...,xq) could be found in 
the separated form 


N 
U(X, X2- -> Xa) = È XPG)XP (x)... XX g)- (3.10) 
i=1 
Consider a problem defined in a high-dimensional space of dimension d for the 
unknown field u(x,,...,Xqg). Here, the coordinates x; denote any usual coordinate 
(scalar or vectorial) related to space, time, and/or any conformational coordinate. 
We seek a solution for u(x,,...,Xq) € Q x ++- x Qg. PGD yields an approximate 
solution in the separated form 


N N d 
U(X, xa) = X Xa) e X a = YT] X05). (3.11) 
i=1 i=1 j=1 

If N,, nodes are used to discretize each coordinate, the total number of PGD un- 
knowns is N -N,,-d instead of the (iy? degrees of freedom involved in standard mesh- 
based discretizations. Thus, the high-dimensional solution is computed by solving a 
number of low-dimensional problems alleviating the so-called curse of dimensional- 
ity involved in high-dimensional models. 

Separated representations within the PGD framework were applied for solving the 
multidimensional Fokker—Planck equation describing complex fluids within the ki- 
netic theory framework in [5, 6]. The solution procedure was extended to nonlinear 
kinetic theory descriptions of more complex molecular models in [86]. In [81] authors 
considered multibead-spring models but used a spectral approximation for represent- 
ing all the functions involved in the finite sums decomposition. A deeper analysis of 
nonlinear and transient models was considered in [8]. Complex fluid models were cou- 
pled with complex flows in [105] and [87] opening very encouraging perspectives and 
pointing out the necessity of defining efficient stabilizations. A first tentative of con- 
vective stabilization was proposed in [56]. Finally, in [34] PGD was applied for solving 
the stochastic equation within the Brownian configuration field framework. 

Multidimensional models encountered in the finer descriptions of matter (ranging 
from quantum chemistry to statistical mechanics descriptions) were revisited in [7]. 
The multidimensional chemical master equation was solved in [13] and the Langer 
equation governing phase transitions was solved in [79]. 

The solution of a parametric problem u(x, t, p4, . . ., yp) (widely considered in the 
present chapter) can be expressed as 


N P 
U(X, t, His- -+ Up) =) XiCOT; (E) [ | Mi (ua) (3.12) 
i=1 k=1 
where parameters are considered as model extra-coordinates. 


Many times the spatial domain Q, assumed three-dimensional, can be fully or 
partially separated, and consequently it can be expressed as Q = Q, x Q, x Q, or 
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Q = Oy, x Q,, respectively. The first decomposition is related to hexahedral domains 

whereas the second one is related to plates, beams, or extruded domains. We consider 

both scenarios. 

— The spatial domain Q is partially separable. In this case the separated represen- 
tation reads 


N 
u(x, z,t) = È XZT, (3.13) 
i=1 
where x = (x,y) € Qy, Z € Q, andt € Q,. Thus, iteration p of the alternated 
directions strategy at a given enrichment step n consists of: 
1. solving in Q, a two-dimensional boundary value problem (BVP) to obtain 
function X?, 
2. solving in Q, a one-dimensional BVP to obtain function VAA 5 
3. solving in Q, a one-dimensional initial value problem (IVP) to obtain func- 
tion TË. 
The complexity of the PGD simulation scales with the two-dimensional mesh used 
to solve the BVPs in Q,,, regardless of the mesh and the time step used in the 
solution of the BVP and the IVPs defined in Q, and Q, for calculating functions 
Z,(z) and T;(t). 
— The spatial domain Q is fully separable. In this case the separated representation 
reads 


N 

u y,z t) = $ XOYWZ(2)T,(0. (3.14) 
i=l 

Iteration p of the alternated directions strategy at a given enrichment step n con- 

sists of: 

1. solving in Q, a one-dimensional BVP to obtain function XP 5 

2. solvingin Q, a one-dimensional BVP to obtain function y? > 

3. solving in Q, a one-dimensional BVP to obtain function ZE 3 

4. solving in Q, a one-dimensional IVP to obtain function T?. 

The cost savings provided by PGD are potentially phenomenal when the spatial 

domain is fully separable. Indeed, the complexity of the PGD simulation now 

scales with the one-dimensional meshes used to solve the BVPs in Q,, Oy, and 

Q,, regardless of the time step used in the solution of the decoupled IVPs in Q,. 


Even when the domain is not fully separable, a fully separated representation could 
be considered by using appropriate geometrical mappings or by immersing the non- 
separable domain into a fully separable one. The interested reader can refer to [54] 
and [51]. 

In-plane-out-of-plane separated representations are particularly useful for ad- 
dressing the solution of problems defined in plates [21], shells [22], or extruded do- 
mains [82]. A parametric three-dimensional elastic solution of beams involved in 
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frame structures was proposed in [23]. The same approach was extensively considered 
in structural plate and shell models in [48, 114-118, 107]. Space separated represen- 
tations where enriched with discontinuous functions for representing cracks in [53], 
delamination in [84], and thermal contact resistances in [38]. Domain decomposition 
within the separated space representation was accomplished in [88] and localized 
behaviors were addressed by using superposition techniques in [12]. 

The in-plane-out-of-plane decomposition was then extended to many other 
physics: Thermal models were considered in [38]; squeeze flows of Newtonian and 
non-Newtonian fluids in laminates in [52]; flows in stratified porous media in [35], 
nonlinear viscoplastic flows in plate domains in [26], and electromagnetic problems 
in [112]. A full space decomposition was also efficiently applied for solving the Navier- 
Stokes equations in the lid-driven cavity problem in [44-46]. 


3.1.3 Illustrating the simplest separated representation 
constructor 


In order to illustrate the simplest procedure for constructing the separated represen- 
tation we consider the one-dimensional heat transfer equation involving the temper- 
ature field u(x,t), 


Le ees 1 
ot ox? f (315) 


defined in the space-time domain Q = Q, x Q, = (0, L) x (0, T]. The diffusivity k and 
source term f are assumed constant. We specify homogeneous initial and boundary 
conditions, i.e., u(x,t = 0) = u(x = 0,t) = u(x = L,t) = 0. More details and more 
complex scenarios can be found in [36]. 

The weighted residual form of (3.15) reads 


fou u 
| í (Z -5 -f) axat =o, (3.16) 


O, XQ 


for all suitable test functions u*. 
Our objective is to obtain a PGD approximate solution in the separated form 


N 
u(x,t) = È XiT). (3.17) 
i=1 


We do so by computing each term of the expansion at each step of an enrichment 
process, until a suitable stopping criterion is met. 

Thus, at enrichment step n, the n — 1 first terms of the PGD approximation (3.17) 
are known: 


n-1 
ue Y XOTA. (3.18) 
i=1 
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We now wish to compute the next term X,,(x)T,,(¢) to get the enriched PGD solution 


n-1 
u(x,t) =u" (x, t) + Xp OTa) = Y XOTA + XOT O). (3.19) 
i=1 
One must thus solve a nonlinear problem for the unknown functions X,,(x) and 
T,,(t) by means of a suitable iterative scheme. The simplest strategy consists of an al- 
ternated direction fixed point algorithm, which at iteration p reads 


u(x, t) = u(x, t) + XP(x)TP(t). (3.20) 


Starting from an arbitrary initial guess T? (t), the alternating direction strategy 
computes X? (x) from TP" (t), and then T?(t) from X? (x). These nonlinear iterations 
proceed until reaching a fixed point within a user-specified tolerance e, i. e., 


[Ro YR(y) -XE T) - YP) I] < €, (3.21) 


where || - || is a suitable norm. The enrichment step n thus ends with the assignments 
X,(x) — X? (x) and T,,(t) — TP(t). 

The enrichment process itself stops when an appropriate measure of error €(n) 
becomes small enough, i.e., €(n) < č. 

For additional details the interested reader can refer to [38], where the problems 
related to the calculation of functions X? and T? were defined, as well as more complex 
scenarios involving two-dimensional and high-dimensional problems. 


3.1.4 Convergence properties 


The convergence of the greedy technique is demonstrated for elliptic linear operators 
in classical separation cases (space/space, parameters/space) [10, 27, 62, 80]. How- 
ever, the estimates of the convergence rate remain crude in the sense that they do not 
reflect what is observed in practice. For eigenvalue problems, convergence properties 
were given in [28]. For nonlinear problems, there are few results. However, the con- 
vergence of PGD is shown for convex problems in the sense that the set of PGD-like 
solutions is dense in the admissible space [63]. 


3.1.5 Verification 


3.1.5.1 A posteriori error estimators and adaptive computational approaches 


As any numerical method, PGD is associated with error sources which need to be effec- 
tively assessed and controlled, using a posteriori error estimation, in order to certify 
the accuracy of the results and then permit the transfer and intensive use of PGD-ROMs 
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in industrial activities for robust optimization and design. This is a main challenge in 
simulation-based engineering as identified in the report of the NSF Simulation-Based 
Engineering Science panel [100]. Moreover, one can resort to adaptive methods based 
on a posteriori error estimation for the construction of the reduced models in the hope 
of reducing the computational cost for a given accuracy. 

Verification of PGD-ROMs has only been addressed in few works this last decade, 
in contrast to the vast literature dedicated to the control of the reduced basis method. 
A first attempt was considered in [9] using residual-based techniques for goal-oriented 
error estimation. In this work, mainly devoted to adaptivity, only the error coming from 
the truncation of the PGD modal representation was controlled and the error bounds 
were not guaranteed (the adjoint solution being approximated with a finer PGD de- 
composition). In [2], the approach of [9] was extended to the nonlinear context by 
using a linearized version of the problem to define the adjoint problem, before using 
a weighted residuals method with a higher number of PGD modes to represent the ad- 
joint solution and catch the PGD truncation error. Even though this approach is cheap, 
it still cannot deliver guaranteed error bounds in which all error sources are taken into 
account. 

In order to provide a general framework to obtain guaranteed, accurate, and fully 
computable bounds to effectively control the quality of PGD approximations, a robust 
a posteriori verification technique based on the constitutive relation error (CRE) con- 
cept [70, 31, 76] has been introduced in [73, 30, 74, 33, 113]. This was done in the context 
of parameterized linear elliptic or parabolic problems, the bounds being related to the 
global error (in the energy norm) or to specific quantities of interest (goal-oriented er- 
ror estimation) using adjoint-based techniques [102, 104, 17, 75]. The error estimates 
involve all error sources including discretization error and truncation error in the PGD 
modal representation. They are based on the construction of perfectly equilibrated 
fields. The key technical point is to construct a finite element-equilibrated stress or 
flux vector from finite element equilibration properties of the computed PGD modes; 
after one uses similar tools as those used in the classical finite element analysis con- 
text. This is the mandatory procedure to recover strict bounds on discretization error 
which can be applied to other ROMs. A variant was also proposed in [85], even though 
equilibrated fields were here obtained using a dual PGD computational approach. Fur- 
thermore, the work in [33, 113] enables to split error sources by means of specific er- 
ror indicators, which helps driving greedy adaptive algorithms to drive computations 
and optimize CPU time and memory space for a prescribed error tolerance; this comes 
down to defining a suitable PGD approximation in terms of the required number of 
terms in the modal representation of the solution, but also in terms of the discretiza- 
tion meshes used to compute modes. Consequently, the verification procedure based 
on CRE certifies the quality of the PGD approximation (globally or on specific outputs 
of interest) over the whole set of possible model parameters, and enables to adapt the 
PGD solution towards the specific goals of the computer simulation. The method was 
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illustrated in [33] with several numerical experiments on two-dimensional and three- 
dimensional mechanical problems; one of them is given at the end of this section. 
Let us note that advection problems were not considered in the previously mentioned 
works, even though these could be extended to such nonsymmetric problems with 
minor changes using ingredients given in [103, 60]. Extension to nonlinear problems 
could also be performed using [72] to get guaranteed bounds. Nevertheless, some CRE 
error indicators which are not guaranteed bounds are easy to compute from [68]. 

Eventually, we mention some other works in which PGD model reduction and ver- 
ification methods are conjointly addressed. In [122], the effect of the separated approx- 
imation of input data in the accuracy of the resulting PGD solution was studied, from 
empirical and numerical considerations. In [31, 4], PGD was used to compute the CRE- 
based error estimate itself. In these latter works, the construction of equilibrated fields 
was facilitated using a PGD representation of the solution at the element level in the 
finite element mesh, parameterizing material properties and element shape. In ad- 
dition to making the implementation of CRE into commercial finite element software 
easier, it was shown that the use of PGD enabled to optimize the verification procedure 
and to get both accurate and reasonably expensive upper bounds on the discretization 
error. 


3.1.5.2 Error-driven PGD computation 


Error analysis and computation can lead to efficient and robust PGD computation 
methods. In [65], the objective was to derive a reduced-order formulation such that 
the accuracy in given quantities of interest is increased when compared to a standard 
PGD method. Contrary to traditional goal-oriented methods that usually compute the 
solution of an adjoint problem following the calculation of the primal solution for error 
estimation and adaptation, it was proposed in this work to solve the adjoint problem 
first, based on a reduced approach, in order to extract estimates of the quantities of 
interest and use this information to constrain the reduced primal problem. This ap- 
proach shares similarities with the work described in [20], where the authors define 
specific norms with additional weighting terms taking into account the error in the 
quantity of interest. The main idea in [20] is to minimize a norm weighted by a func- 
tional involving the adjoint solution, via a penalization approach, in order to obtain a 
goal-oriented PGD using the so-called ideal minimal residual approach. 

Eventually, a new and promising PGD computational method based on the min- 
imization of the CRE measure was proposed and analyzed in [3, 32]. In addition to 
enhancing the computation of PGD modes, it provides an improved, immediate, and 
robust reduction error estimation. This technique has been extended to solid mechan- 
ics nonlinear time-dependent problems [77]. 
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3.1.5.3 Illustration 


We present here a three-dimensional numerical experiment taken from [33], which 
illustrates the error estimation method and adaptive strategy. One considers a three- 
dimensional elasticity problem with three parameters for which the first PGD modes 
are computed; the associated errors as well as specific error indicators related to the 
discretization and the number of PGD modes are also given. 

The domain is a cube of size 1m x 1m x 1m with three spherical inclusions for 
which Young’s moduli E; € [1,10] (1 < i < 3) are parameters, so that the order m 
PGD representation reads u,,(Xx, E1, E>, E3). The three inclusions have the same radius 
r = 0.1m, and their centers are respectively located at points c} = (0.2, 0.2, 0.2), cy = 
(0.6, 0.3, 0.5), and c3 = (0.4, 0.7, 0.8) (Figure 3.1). The cube is clamped along the plane 
located at x = O and subjected to a unit traction force Fz = +x applied on the plane 
located at x = 1. The initial finite element mesh contains 17,731 4-node tetrahedral 
elements and 3,622 nodes (10,866 degrees of freedom). 


Figure 3.1: Three-dimensional elasticity problem: space domain with three inclusions and associated 
finite element mesh. 


The first three PGD modes of the PGD approximate solution uh, are given in Figure 3.2 
for space functions p(X) and in Figure 3.3 for parameter functions y; m(E1), Y2m(E2)» 
and y3 m(E3). Note that the first space function p(x) corresponds to a global mode, 
whereas the second and third space functions p, (x) and w(x) are local modes mostly 
concentrated around the first and second inclusions, respectively. 

The evolutions of the CRE-based error estimate Ecçgg and associated error indica- 
tors Npgp and qis with respect to the number m of PGD modes are shown in Figure 3.4 
for m = 1,...,6 and for the maximal values obtained with triplets (E4, E2, E3). This is 
represented along the adaptive strategy, and vertical evolutions indicate mesh refine- 
ments (which are performed each time the indicator associated with the discretiza- 
tion error is larger than the one associated with the PGD truncation error). Let us note 
that the error estimate converges quite fast toward the indicator associated to the dis- 
cretization error (for a fixed mesh), while both error indicators decrease toward zero 
along the adaptive procedure. The computation cost associated with the error estima- 
tor and error indicators is of the same order as for the PGD computation. 
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Figure 3.2: Magnitude of space functions p(x) obtained for order m = 1,...,3 (from top to bottom). 
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Figure 3.3: Parameter functions yı m(E1), Y2,m(E2), and 73, (E3) (from left to right) obtained for order 
m = 1,...,3 (from top to bottom). 
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3.1.6 Limits 


PGD limits coincide with the limits of the variable separation hypothesis. Generally 
speaking, such limits could be reached by problems with moving loads. A thorough 
analysis was performed for a specific model problem in [3]; it follows that the quasi- 
stationary solution is not time-space separable with few modes. However, adding in 
the basis this particular solution (which is generally easy to compute), an accurate 
approximation can be obtained with few PGD modes. Transient dynamics also belongs 
to this class of problems where the time-space variable separation may not work well, 
in particular when high-frequency phenomena occur. That is clear, as considering a 
wave implies that the driven quantities are a combination of time and space variables. 
However, working over the frequency-space domain still enables to derive efficient 
PGD approximations [16]. 


3.2 PGD for nonlinear time-dependent problems 


3.2.1 State of the art 


Large time increment (LATIN)-PGD is a robust and effective tool for the construction of 
PGD in nonlinear solid mechanics, in the process of implementation in the industrial 
simulation codes; elsewhere, the approach it underlies should be extended to other 
parts of physics. The article [77] and the book [68] describe the state of the art. Until 
recent years, LATIN-PGD had no competitor except [110, 111], where POD is used in 
conjunction with a “hyperreduction” technique. Currently, many works using PGD or 
POD have appeared in the frame of the classical homogenization method for periodic 
media (in particular finite element square), among them [108, 121, 64, 59]. 
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3.2.2 The LATIN-PGD computation method 


The LATIN-PGD computation method has been derived to build ROMs in nonlinear 
solid mechanics but could be extended to other parts of physics. 


3.2.2.1 Presentation of the problem 


To present the method, with the assumption of small perturbations, let us consider 
the quasi-static and isothermal evolution of a structure defined over the time-space 
domain [0, T] x Q. This structure is subjected to prescribed body forces f 4, traction 
forces F, over a part 0,0 of the boundary, and displacements uq over the complemen- 
tary part 0,Q (Figure 3.5). 


The state of the structure is defined by the set of fields s = (¿pX , 0, Y) (where the dot 

notation Ġ denotes the time derivative), in which: 

— £p refers to the inelastic part of the strain field € which corresponds to the displace- 
ment field u, uncoupled into an elastic part £, and an inelastic part £p = £ — £e; 
X refers to the remaining internal variables; 

— Ø refers to the Cauchy stress field and Y to the set of variables conjugate of X 
(Y and X have the same dimension); X could be hardening variables, damage 
variables, chemical variables, etc. 


All these quantities are defined over the time-space domain [0, T] x Q and assumed to 
be sufficiently regular. For the sake of simplicity, the displacement u alone is assumed 
to have a nonzero initial value, denoted uy. Introducing the notations for the primal 
fields 


e = |*e | T H and e,= | so that e, = € - ee (3.22) 
and for the dual fields 


f= Hi (3.23) 
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the mechanical dissipation rate for the entire structure Q is 


[è :0-X- Yaa = [(@ ao, (3.24) 
Q Q 


where - denotes the contraction adapted to the tensorial nature of X and Y. Notation o 
denotes the contraction operator for generalized quantities. Let us now introduce the 
following fundamental bilinear “dissipation” form: 


(s,s') = | (1 5 Fle, of’ +e), o f)dOdt, (3.25) 
[0,T]xQ 


along with E and F, the spaces of the fields é, and f which are compatible with (3.25). 
These spaces enable us to define S = E x F, the space in which the state s = (ép, f) of 
the structure is being sought. 

Following [67, 68], a normal formulation with internal state variables is used to 
represent the behavior of the material. If o denotes the mass density of the material, 
from the free energy p'¥(€,, X) with the usual uncoupling assumptions, the state law 
yields 


ow oy 
=p— =K Y = p— = AX, 2 
o=p E. £. and Pay (3.26) 


where Hooke’s tensor K and the constant, symmetric, and positive definite tensor A 
are material characteristics. 


The state evolution laws can be written 
èp = Bf) with pio = 0, (3.27) 


where B is a positive operator which is also for most viscoplastic models maximal 
monotone. Let us introduce now the space uo of admissible displacement fields 
u defined over [0, T] x O and us the associated vectorial space. The compatibility 
equation can be written as follows: 


* [0,7] 
such that Vu" € Ug do 


Tr[e(u)Ke(u")|dQat = | Tr[e,Ke(u")|dQdt 
[0,T]xO [0,T]xQ 
{ | fq-u*dQdt + | F,-u'dSdt. (3.28) 
[0,T]xQ [0,T]xa,Q 


; [0,T] 
Find u € Und 


It follows that the stress ø = K(e(u) — Ep) can be written 


0 = QE, +ra (3.29) 


112 — FF. Chinesta and P. Ladevéze 


where Q is a linear given operator and r, is a prestress depending on the data. Intro- 
ducing the generalized stress, the admissibility conditions can be written as 


f= Qe, +¥q (3.30) 
with 
7 QO O = Yq 
Q= j A and r= AI (3.31) 


where Q is a linear symmetric positive operator. Finally, the problem to solve is 


Find s = (€,,f) € s!®T! such that 


: . (3.32) 
f= Qe, + ra and è = Bf) with epi-0 = 0. 
Consequently, one has to solve a first-order differential equation with an initial 
condition. The operators Q and B as well as the right-hand side member r4 could de- 
pend on the parameter u belonging to the parameter set £. 


3.2.2.2 The solver LATIN for ROM computation 


Let us consider ROM computations based in time and space separation. A natural and 
general idea is to transform the reference problem into a succession of linear global 
problems over [0, T]xQ which could depend on parameters. Using reduced basis, POD, 
or PGD, an ROM can be built over [0, T] x Q for each linear problem. The final ROM 
is then obtained gathering all the previous ROMs. The LATIN method is an iterative 
strategy which differs from classical incremental or step-by-step techniques in that, 
at each iteration, it produces an approximation of the full structural response over 
the whole loading history being considered (see Figure 3.6). In other words, the name 
LATIN was not chosen very well because the method is essentially nonincremental. 


Classical step-by-step methods LATIN method 
Idea: performance #4 = mechanics-hased | 


Figure 3.6: The LATIN method and classical step-by-step methods. 
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The LATIN method, which operates over the time-space domain [0, T] x Q, is very con- 

venient for solving the reformulation (3.32) of the reference problem. Its first principle 

consists in separating the difficulties. Thus, the equations are divided into: 

— aset of linear equations which can be global in the space variables: the equilib- 
rium and compatibility equations and the state equations; 

— aset of equations which are local in the space variables but can be nonlinear: the 
state evolution laws. 


Local stage at iteration n + 1 (see Figure 3.7) 
Find $41/2 = (€pnt/2 fn+1/2) € s°") such that 


Cnnit/2 = Blfstj2) with €@, 44/2 = Oat t = 0, (3.33) 


Èpn+1/2 = Cnn +H* (En = f,) = 0. 

The search direction H* is a parameter. Practically, one takes a linear positive op- 
erator which is local in both time and space variables. It follows that the problem to 
solve is local in the space variable and then can be split into small independent prob- 
lems associated to Gauss points. This local stage is very suitable for parallel comput- 
ing. 


(0, Y) r 


_ Figure 3.7: The geometric representation associated to the 
(pX) reformulation of the reference problem. 


_ Figure 3.8: Iteration n + 1 ofthe LATIN method over 
X) [0,7] x Q. 


Linear stage at iteration n + 1 (see Figure 3.8) 
Find Sp, = (pnt fny1) € SI>” such that 


frst = Qe nit +Y¥q, 


i z 4 F (3.34) 
@p,n+1 ~ Èp,n+1/2 7 H (fm - fn+1/2) =0 with epn = Oatt=0. 


The search direction H` is a parameter. This is a linear positive operator which is 
local in both time and space variables. It is associated to the material operator B. One 
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has to solve a first-order linear differential equation with an initial condition, the oper- 
ator Q being nonexplicit. In practice, H` is chosen close to the tangent to the manifold 
T at the point §,,4/2 = (Gp ns1/2» f, 41/2)+ For H*, one takes 0 or H`. The convergence of 
the iterative process has been proved in the case of nonsoftening materials and con- 
tacts without friction [68]. Precisely, the iterative process converges if: 

— the material operator B is maximal monotone; 

— the material operator A is positive definite; 

— the search directions H` and H* are positive definite and equal H` =H. 


The distance between two successive approximations gives a good and easily com- 
putable error indicator. Let us also note that one often uses an additional relaxation 
with a coefficient equal to 0.8. 


Let us introduce corrections: 


Ae, = Cp nt = enw 


(3.35) 
Af = fri - fr 


where Sp = (p,m fn) has been computed at iteration n. The problem to solve over 
[0, T] x Q at iteration n + 1 is then: 


Find As = (Ae,, Af) € s!°7] such that 
Af = QAé, (3.36) 
Ae, -H Af=R, with Ae, =Oatt=0. 


Problem (3.36) is interpreted as a linear constitutive relation, the operator H` be- 
ing local in both time and space variables and positive definite as the Hooke tensor. 
Consequently, one introduces the associated CRE which defines the global residual to 
minimize 


r(As, t) = : fiae, - HAF -R,](H7) [Aè - H7Af — R,]do (3.37) 
Q 
and 
R(As) = | (1 — z ribs t)dt (3.38) 
= }r(hs, l 
[0,T] 


with As = (Ae, Af) € s[®T], The problem (3.36) becomes 


Find As e S!®7] miminizing 
As € $'°7] > R(s) € R (3.39) 
with the constrains Af=QAe, and Aẹ,=0att= 0. 
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The time residual r(As, t) can be used to build ROMs with the reduced basis 
method. One only prescribes that 


Ae, = Z AOS (3.40) 


with A,(0) = 0 (initial condition), g; € L’(Q), and A,(t) € L’[0, T]. It follows, using 
admissibility conditions, that 


Af = ¥'Aj(t)Qg;(x), (3.41) 
i=1 


where Qg;(x) are computed solving several elasticity problems. 


3.2.2.3 Minimization technique 


Let us start with 


s, = (ė?, £) + F (AB), AP). (3.42) 


The iteration n + 1 has two steps. 


Step 1: Updating of the PGD time functions - This POD phase relies on the space 
PGD modes (EŻ, F') for which the computation cost is relatively high. New time func- 
tions, still noted A;, are computed minimizing the residual R with the constraint A,(0) = 
0. One gets a small system of differential equations over the time interval with con- 
ditions at both ends. The problem can be also solved globally over the time interval 
[0, T]. 


Step 2: Addition of anew PGD mode — One computes following a “greedy” algorithm 
S41 = Sn + (ÀE, AF) (3.43) 


with A(O) = 0. The additional PGD mode is obtained through the minimization of the 
residual R, alternatively on the time function A and on the space function E,,. The ini- 
tialization of this iterative process is done taking as the first time function guess the 
root square of the time residual r(0, t). The minimization with respect to the space 
variables leads to the resolution of a time-independent spatial problem defined over 
Q; thatis, a classical finite element problem. The minimization with respect to the time 
variable leads to a scalar differential equation over [0, T] with conditions at both ends 
whose resolution is quite inexpensive; the easier way is to solve the global time prob- 
lem coming from the residual minimization. The iterative process is stopped after few 
iterations, practically two or three. Let us also note that this second step is canceled if 
the residual R(O) is relatively small. 
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3.2.3 Illustration 


The previous strategy has been applied in numerous cases. To illustrate the perfor- 
mances, we consider the engineering example presented in [90] and developed in 
collaboration with SAFRAN. This is a relatively small case with 151,600 degrees of 
freedom and 60 time steps. The geometry is freely inspired from a blade of the Vul- 
cain engine (Figure 3.9) and the material behavior is the Marquis—Chaboche elastic- 
viscoplastic law (with kinematic hardening and Norton power law). The problem is 
not only strongly nonlinear and time-dependent. Two material parameters (power y 
and yield stress Rọ), as well as the loading amplitude, are not very well known. The 
range of variation of each parameter was discretized into 10 arbitrarily values, leading 
to 1,000 different nonlinear time-dependent problems. 


Symmetry 
condition 


Displacement (mm) 


120 time (s) 


Figure 3.9: Geometry, boundary conditions, and mesh of the blade test case. 


For nonlinear time-dependent problems involving especially viscoplasticity and dam- 
age, LATIN-PGD has been easily extended to take into account material or loading pa- 
rameters which are seen as extra-coordinates. Reduced models are built on the param- 
eters/time/space separation [119, 120]. For few parameters, it could be advantageous 
to describe point-by-point the parameter space using the remarkable property of the 
LATIN method: The initialization of the iterative process can be any function defined 
over [0, T] x Z, [90]. 

The computations have been carried out on an Intel bi-Xeon processor (total of 12 
cores) at 2.8 GHz with 12 GB of RAM. Figure 3.10 gives the virtual chart related to the 
maximum value of the Von Mises stress. The computational time is about 25 days (es- 
timated time) to complete the 1,000 resolutions with ABAQUS and less than 17 hours 
with the LATIN-PGD method, leading to a gain of more than 35. 

In this example, material parameters (Rọ, y) were assumed to be stochastic, 
whereas the loading amplitude was assumed to be defined by its interval of varia- 
tion. The use of the previous virtual chart allows to deal with uncertainties in an 
inexpensive manner. For example, one can build the interval of variation of the max- 
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imum value of the Von Mises stress with stochastic bounds, like the result presented 
in Figure 3.11. 


3.2.4 Additional reduction or interpolation 


Let us note that LATIN-PGD needs to compute numerous integrals as 


I= | | MH, Mgt, Madar, (3.44) 
0T] à 


where H changes along the iterations, as it can be time-dependent and also nonlinear 
in terms of the computed solution; f and g are not necessarily represented in the PGD 
framework. It follows that the computation requires to loop on all the time steps and 
all the space Gauss points and consequently its cost could be high. For ROM computa- 
tions where such integral computations are performed online, this problem is a crucial 
one. This is not the case for LATIN-PGD, where all these calculations are done offline; 
however, it is always interesting to reduce the computation cost. Several additional re- 
duction or interpolation methods have been proposed to overcome this difficulty and 
are described in this book, including the empirical interpolation method (EIM), the 
discrete EIM (DEIM), hyperreduction, etc. Here, we introduce another recent method, 
named “reference point method” (RPM) [39, Chapter 3], [29]. 
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Let us divide the time interval J = [0, T] being studied into m subintervals {J;};-1, am 
of lengths {At;}-1,m as Shown in Figure 3.12. Introducing the centers {t;};-1,_m of these 
subintervals, called “reference times,” one has J; = [t; — At;/2, t; + At;/2]. In the space 
domain, let us also introduce m’ points {Mj} ;-1,.,.m' and partition Q into {Q;})-1 im’, as 
shown in Figure 3.12. These points are called “reference points” and the measures of 
the subdomains are denoted {w;}i-j,.„m'- 


ti bh ti tm 
Figure 3.12: The reference times over 
[0, T] and the reference points over Q. 


Let us consider that one needs 20 modes to describe the solution. One can take dou- 
ble space reference points, i. e., 40, and for the reference times the minimum between 
40 and the number of time degrees of freedom. The choice of the reference times and 
reference points is unrelated to the classical discretizations of the time interval and 
space domain. Refined time and space discretizations should still be used for the cal- 
culation of the various quantities. Here, our purpose is to describe a field f over the 
time-space domain [0, T] x Q through 


f(t, M) ifte, 
(0) otherwise 


f(t, M) ifMe QO, 
(0) otherwise, 


da= and ban = | (3.45) 


I 
m are the generalized com- 


ponents of f. One should note that these quantities verify the following compatibility 


conditions: for i = 1,...,m andj = 1,...,m', 


with i = 1,...,m andj = 1,...,m’. The sets a, AS 


@i(t;) = biM,). (3.46) 


It could happen that the quantity f is not well represented over the time-space do- 
main. Then, one adds if necessary a PGD description of the residual. The extension to 
parameter-dependent functions is easy. The great interest of such generalized compo- 
nents is that operations (addition, multiplication, derivation) are greatly facilitated. 
Then, the main question is: How can one build or rebuild a field from its components? 
We choose to define function f from its components using only one product per time- 
space subdomain J; x Qj: 


f(t, M) : PODM) Y(t, M) € I, x Q. (3.47) 
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The following very simple formula follows from [29]: 


m' akinak 
Èk wai (t)a; (t;) ÈM), (3.48) 


f(t,M) : a;(t)b;(M) = ma ae =p i 
Èp- Wâ; (6) a; (ti) 


where the quantities â, f, and its approximation are equal for the time-space points 
(t, M A) of the time-space domain. In [29], one will find a performance analysis as well 
as comparisons with the EIM. 


3.2.5 Extensions 


LATIN-PGD has been developed for most structural mechanics problems [68] and ro- 

bust ROM computational methods are available for several important issues: 

- Cyclic viscoplasticity and fatigue for engineering structures [40, 41, 19]. 
APGD with two time scales has been introduced and developed for cyclic loadings. 
The applications deal with small displacement problems involving (visco)plastic 
and damageable materials. 

- Large displacement problems with instabilities [68, 15, 24, 25]. PGD has been 
extended to large displacement problems for which the classical time/space sep- 
aration hypothesis does not work or does not work well. The key was a new 
“material” reformulation of the structure problem; applications deal with large 
(visco)plastic deformation problems and elastic buckling problems. 

— Concurrent multiscale and multiphysics problems for nonlinear time-de- 
pendent problems [89, 69, 71, 92, 47]. Nonlinear time-dependent problems are 
considered under the small displacement hypothesis. The key here is a mixed do- 
main decomposition method with two scales over the time-space domain. Classi- 
cal PGD is used to solve at each iteration the micro-problems. A further path in 
[43] is to build a PGD-ROM for the computation of the micro-problems over the 
complete time interval. For multiphysics problems, the idea consists in introduc- 
ing an abstract interface between physics which plays the same role as the usual 
interface material. 


For all these issues, the LATIN-PGD version described in this chapter should be a 
paradigm. 


3.3 Parametric solutions 


This section illustrates how parameters of different natures become coordinates. The 
problems considered are quite simplistic but the same rationale is considered for solv- 
ing more complex problems reported at the end of the section. We consider three types 
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of parameters: (i) parameters related to the model; (ii) parameters related to initial 
and boundary conditions; and (iii) geometrical parameters defining the space-time 
domain in which the model is defined. 


3.3.1 Model parameters as extra-coordinates 


We consider the parametric heat transfer equation 


ou 
— —kAu-f =0, . 
a u-f (3.49) 


with homogeneous initial and boundary conditions. Here (x,t, k) € Q% x Q, x Qg. The 
scalar conductivity k is here viewed as a new coordinate (called extra-coordinate) de- 
fined in the interval Q}. Thus, instead of solving the thermal model for different dis- 
crete values of the conductivity parameter, we wish to solve only once a more general 
problem. For that purpose we consider the weighted residual form related to equa- 
tion (3.49): 


| ut ( a - kdu- f Jax dt dk = 0. (3.50) 
OXQ; XQ, 


The PGD solution is sought in the form 
N 
u(x,t, k) = X X\(x)T;(t)K;(k), (3.51) 
i=1 


constructed using rank-one updates and the alternate directions fixed point algorithm 
for addressing the nonlinearity, as discussed in the previous section. 


3.3.2 Boundary conditions as extra-coordinates 
For the sake of simplicity we first consider the steady-state heat equation 
V -(K- Vu(x)) + f(x) = 0, (3.52) 


with x € Q c R’, subjected to the boundary conditions 


| u(x € Ty) = Ug, (3.53) 


(-K- Vu)lxer, ` N = qg : N = qg, 


with K being the conductivity tensor and n the outwards unit vector defined in the 
domain boundary T„, with oO =T =T4 Ur andTa NT, = 4. 

In what follows we consider the simplest scenario that consists of constant Neu- 
mann and Dirichlet boundary conditions. More complex and general situations were 
addressed in [37], where nonconstant boundary and initial conditions were addressed. 
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3.3.2.1 Neumann boundary condition as extra-coordinate 


First, imagine that we are interested in the model solution for values of the heat flux 
Gg € Tq = [ag> qzl. We could consider the given heat flux as an extra-coordinate and 
then solve only once the resulting four-dimensional heat equation for calculating the 
general parametric solution u(x, qg). For this purpose the solution is sought in the sep- 
arated form 


N 
U(X, qg) = È Xi) - Q;(qg). (3.54) 
i=1 
In order to enforce the prescribed Dirichlet boundary condition u(x € T4) = ug, 
the simplest procedure consists of choosing the first functional couple X;(x)-Q,(q,) in 
order to ensure that u! (x € Ta qg) = X(x € T4): Q1(qg) = Ug. Thus, the remaining terms 
of the finite sum X;(x), i > 1, will be subjected to homogeneous essential boundary 
conditions, i. e., X;(x € T4) = 0. 
In order to use the approximation (3.54) we start by considering the weak form 
related to equation (3.52), which reads as follows: Find u(x) regular enough, verifying 
u(x € Ty) = Ug, such that 


| Vu" - (K - Vu) dx = | u* (K-Vu)-ndx+ | u* f(x) dx (3.55) 
Q I, Q 


is verified Vu", with u* (x € Ty) = 0. 
By introducing the Neumann condition (3.53) into (3.55) we obtain 


| vu" - (K - Vu) dx = — | us qg AX + fu f(x) dx, (3.56) 
Q I, Q 


which allows constructing the separated form (3.54) using the rank-one updates and 
the alternate directions fixed point algorithm for addressing the nonlinearity. 


3.3.2.2 Dirichlet boundary condition as extra-coordinate 


Now we consider the solution of model (3.52) for any value of ug in (3.53) in a certain 
interval Z, = [u,, Ug]. For this purpose we consider the function g(x) continuous in Q 
such that Ag € L,(Q) and g(x € T4) = 1. Thus, we can define the change of variable 
[54]. We have 


U(X) = V(X) + Ug P(X), (3.57) 
which allows rewriting equations (3.52) and (3.53) as 


V - (K- Vv(x)) + Ug (K - Vp(x)) + f(x) = 0, (3.58) 
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which proceeding from its weak form allows again constructing the separated repre- 
sentation of the solution 


N 
V(X, ug) = È X;(x)Ui;(ug). (3.59) 
i=1 


3.3.3 Parametric domains 


For the sake of clarity and without loss of generality we address in this section the 
transient one-dimensional heat equation 


ðu u 
— =k +f, 3.60 
a ae eo 
with t € Q, = (0,0], x € Q, = (0, L), constant conductivity k and source term f, and 
homogeneous initial and boundary conditions, i. e., u(x = 0, t) = u(x = L, t) = u(x,t 
0) = 0. 

The associated space-time weak form reads 


ot 
O, xO, Q XQ; QO, XQ; 


| eet pape’ | pe ay | u* f dx dt. (3.61) 
Ox Ox 


If we are interested in computing the solution u(x, t) in many domains of length 
L € Q; = [L’,L*] and for many time intervals of length © € Og = [07, 0°], more than 
solving the model for many possible choices, it is preferable to compute the parametric 
solution by considering L and © as extra-coordinates. However, equation (3.61) does 
not involve an explicit dependence on the extra-coordinates L and O, both defining 
the domain of integration. In order to make this dependence explicit, we consider the 
coordinate transformation 


t=T0, teTZ= [0,1], 
.62 
ee AeZ = [0,1]. (362) 
In this case the weak form (3.61) reads 
«OU 2 du” du © , 
| u AL dA at = | ke hae + | u*fLO dA dr, (3.63) 
TXE. EE TXT 


which allows calculating the parametric solution u(r, A, L, ©) by considering the sep- 
arated representation 


N 
u(A,t,L, 0) = X X\(A)T;(1) L,(L) TF). (3.64) 


i=1 
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3.3.4 Related works 


This kind of parametric modeling was widely addressed in a panoply of applications, 
where material parameters [106, 11, 78, 21, 1, 18], initial conditions [55, 57], boundary 
conditions [49, 50, 91, 58], and parameters defining the geometry [98, 14, 122] were 
considered as extra-coordinates within the PGD framework. All these parametric so- 
lutions were successfully employed for performing real-time simulations (e. g., surgi- 
cal simulation involving haptic devices addressing contact, cutting, etc.), material ho- 
mogenization, real-time process optimization, inverse analysis, and simulation-based 
control. They were also employed in dynamic data-driven application systems. In [119] 
authors adopted the just referred space-time-parameter separated representation for 
constructing parametric solutions. Other applications of optimization and dynamic 
data-driven application systems are found in [42, 83, 109]. 

For the treatment of the nonlinearities involved in the works just referred, the sep- 
arated representation constructors were combined with numerous nonlinear solvers 
ranging from the most standard ones (fixed point, Newton, etc.) to less standard ap- 
proaches based on LATIN, the asymptotic numerical method (e. g., [91, 82], among 
many others), or the DEIM [37]. 

In the context of stochastic modeling, PGD was introduced in [93] for the uncer- 
tainty quantification and propagation. The interpretation of the separated representa- 
tion constructor as a generalized eigenproblem allowed to define dedicated algorithms 
inspired from solution techniques for classical eigenproblems [94]. In this context de- 
terministic and stochastic contributions were separated, making PGD a promising al- 
ternative to traditional methods for uncertainty propagation, as discussed in [95]. PGD 
was also extended to stochastic nonlinear problems in [96]. More recently, the PGD 
was successfully applied to the solution of high-dimensional stochastic parametric 
problems, with the introduction of suitable hierarchical tensor representations and 
associated algorithms [97]. 

In engineering problems, the classical separated variable representation is used 
for up to around 20 parameters. To go further, a parameter-multiscale PGD has been 
devised to overcome this major limitation of the classical ROM computational tech- 
niques [101]. It is based on Saint-Venant’s principle, which highlights two different 
levels of parametric influence. 


3.4 Space separated representations 


Plates and shells are very common in nature and thus they inspired engineers to use 
both of them from the very beginning of structural mechanics. Nowadays, plate and 
shell parts are massively present in most engineering applications. 
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This type of structural elements involves homogeneous and heterogeneous mate- 
rials, isotropic and anisotropic, linear and nonlinear. The appropriate design of such 
parts consists not only in the structural analysis of the parts for accommodating the 
design loads, but also in the analysis of the associated manufacturing processes be- 
cause many properties of the final parts depend on the formation process itself (e. g., 
flow-induced microstructures). Thus, fine analyses concern both the structural parts 
and their associated formation processes. 

In general the whole design requires the solution of some mathematical models 
governing the evolution of the quantities of interest. These models consist of a set of 
partial differential equations combining general balance equations (mass, energy, and 
momentum) and some specific constitutive equations depending on the considered 
physics, the last involving different material parameters. These complex equations (in 
general nonlinear and strongly coupled) must be solved in the domain of interest. 

When addressing plate or shell geometries the domains in which the mathemat- 
ical models must be solved become degenerated because one of its characteristic di- 
mensions (the thickness in the present case) is much lower than the other characteris- 
tic dimensions. We will understand the consequences of such degeneracy later. When 
analytical solutions are neither available nor possible because of the geometrical or 
behavioral complexities, the solution must be calculated by invoking any of the avail- 
able numerical techniques (finite elements, finite differences, finite volumes, methods 
of particles, etc.). 

In the numerical framework the solution is only obtained in a discrete number 
of points, usually called nodes, distributed in the domain. From the solution at those 
points, it can be interpolated at any other point in the domain. In general, regular 
nodal distributions are preferred because they offer better accuracies. In the case of 
degenerated plate or shell domains one could expect that if the solution evolves signif- 
icantly in the thickness direction, a large enough number of nodes must be distributed 
along the thickness direction to ensure the accurate representation of the field evolu- 
tion in that direction. In that case, a regular nodal distribution in the whole domain 
will imply the use of an extremely large number of nodes, with the consequent impact 
on the numerical solution efficiency. 

When simple behaviors and domains were considered, plate and shell theories 
were developed in the structural mechanics framework allowing, through the intro- 
duction of some hypotheses, reducing the three-dimensional complexity to a two- 
dimensional one related to the problem now formulated by considering the in-plane 
coordinates. 

In the case of fluid flows this dimensionality reduction is known as lubrication 
theory and it allows efficient solutions of fluid flows taking place in plate or shell ge- 
ometries for many type of fluids, linear (Newtonian) and nonlinear. The interest of 
this type of flows is not only due to the fact that it is involved in the manufacturing 
processes of plate and shell parts, but also due to the fact that many tests for charac- 
terizing material behaviors involve it. 
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However, as soon as richer physics are included in the models and the considered 
geometries differ from those ensuring the validity of the different reduction hypothe- 
ses, simplified simulations are compromised and they fail in their predictions. 

In these circumstances the reduction from the three-dimensional model to a two- 
dimensional simplified one is not obvious, and three-dimensional simulations appear 
many times as the only valid route for addressing such models, which despite the fact 
that they are defined in degenerated geometries (plates or shells), they seem to re- 
quire a fully three-dimensional solution. However, in order to integrate such a calcu- 
lation (fully three-dimensional and implying an impressive number of degrees of free- 
dom) in usual design procedures, a new efficient (fast and accurate) solution proce- 
dure is needed. The in-plane-out-of-plane separated representations represent a valu- 
able route able to compute the different unknown three-dimensional fields without 
the necessity of introducing any hypothesis. The most outstanding advantage is that 
three-dimensional solutions can be obtained with a computational cost characteristic 
of standard two-dimensional solutions, as previously described. In what follows we 
formulate different physics within such a separated representation framework. 


3.4.1 Heat transfer in laminates 


In this section we illustrate the construction of the PGD of a generic model defined in 
a plate domain E = Q x Z with Q c R’ and Z = [0, H] c R. For the sake of simplicity 
we consider the model related to the steady-state heat conduction equation 


V-(K- Vu) = 0, (3.65) 


in a plate geometry that contains P plies in the plate thickness. Each ply is charac- 
terized by its conductivity tensor K;(x, y) which is assumed constant through the ply 
thickness. Moreover, without any loss of generality, we assume the same thickness h 
for the different plies constituting the laminate. Thus, we can define a characteristic 
function representing the position of each ply i = 1,...,P: 


1 Z<Z<Ziy1, 


O otherwise, (3.66) 


Xi(Z) = | 
where z; = (i— 1)h defines the location of the i-th ply in the laminate thickness. Now, 
the laminate conductivity can be given in the following separated form: 

i=P 
K(x, y,Z) = ) K(x) -x;@2), (3.67) 


i=1 


where x denotes the in-plane coordinates, i. e., x = (x,y) € Q. 
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The weak form of equation (3.65), with appropriate boundary conditions, reads 


| vu" -(K- Vu) dE = 0, (3.68) 


(1] 


with the test function u* defined in an appropriate functional space. The solution 
u(x, y, Z) is then searched under the separated form: 


=N 
u(x,z) = ) X;(x)Z;(z). (3.69) 
jel 


3.4.2 Three-dimensional Resin Transfer Moulding 


We now summarize the application of PGD to the modeling of resin transfer moulding 
processes. We consider the flow within a porous medium in a plate domain € = Ox T 
with Q c R? andZ = [0,H] c R. The governing equation is obtained by combining 
Darcy’s law, which relates the fluid velocity to the pressure gradient, 


v=-K.-Vp, (3.70) 
and the incompressibility constraint, 
V-v=0. (3.71) 


Introduction of equation (3.70) into equation (3.71) yields a single equation for the 
pressure field: 


V-(K- Vp) = 0. (3.72) 


The mould contains a laminate preform composed of P different anisotropic plies 
of thickness h, each one characterized by a permeability tensor K;(x, y) that is assumed 
constant through the ply thickness. We define a characteristic function 


1 Zi SZ SZ 


3.73 
O otherwise, (3473) 


Xi(Z) = | 


where z; = (i — 1)h is the location of the i-th ply in the plate thickness. The laminate 
permeability is thus given in separated form as follows: 


P 
K(x,y,z) = ) Kix) -X;(2), (3.74) 


i=1 


where x denotes the in-plane coordinates, i. e., x = (x,y) € Q. 
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The weak form of equation (3.72) reads 


Vp" -(K- Vp) dE =0, (3.75) 


E es, 


for all test functions p* selected in an appropriate functional space. Dirichlet bound- 
ary conditions are imposed for the pressure at the inlet and outlet of the flow domain 
p(x € Tp) = p8), while zero flux (i. e., no flow) Vp -n = O is imposed elsewhere (n 
being the unit outwards vector defined on the domain boundary) as a weak boundary 
condition. We seek an approximate solution p(x, y,z) in the PGD form 


N 
pz) = ) X0 -Z(2), (3.76) 
j=l 


which is constructed by using the standard procedure previously discussed. 


3.4.3 The elastic problem defined in plate domains 


We proposed in [21] and original in-plane-out-of-plane decomposition of the three- 
dimensional elastic solution in a plate geometry. The elastic problem was defined ina 
plate domain E = Q x Z with (x,y) € Q, Q c R’ andz € Z, Z = [0, H] c R, H being the 
plate thickness. The separated representation of the displacement field u = (uy, U2, u3) 
reads 


u(y) \ u / Piy) THe) 
u%wyz)=| wy) |=| Rey De f (3.77) 
u3 (x,y, Z) PEAY P3(x,y) T32) 


where Pi, k = 1,2,3, are functions of the in-plane coordinates (x,y), whereas Ti, k= 
1, 2,3, are functions involving the thickness coordinate z. In [21] we compared the first 
modes of such separated representations with the kinematic hypotheses usually con- 
sidered in plate theories. 

Expression (3.77) can be written in a more compact form by using the Hadamard 
(component-to-component) product: 


N . . 
u(x, y, z) = ) P'(x,y) o T(z), (3.78) 


i=1 
where vectors P’ and T’ contain functions PÌ and TÌ respectively. 


Let us consider a linear elasticity problem on a plate domain = = Q x Z. The weak 
form using the so-called Voigt notation reads 


few)’ -K-e(u) dx = fu -fj dx + | u’-F,dx, Vu", (3.79) 


a 2 Ty 
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where K is the generalized 6x6 Hooke tensor, f4 represents the volumetric body forces, 
and F, represents the traction applied on the boundary I',. The separation of variables 
introduced in equation (3.77) yields a separated representation for the derivatives of 
the displacement components u;, i = 1,2,3, and from it the separated representation 
of the strain tensor €: 


apk nk 
zx N 
aP; rk 
oy > 
N pk. 25 
3 
e(u(xy,z)) =Y | ap i , (3.80) 
mo Tereny 
k=1 wy “lita 2 
OPS nk | pk ark 
Dx T3 + Pi KA 
oP; k k ôT; 
ay T; + P3 ETA 


which introduced into the weak form allows computing the separated form (3.78). 


3.4.4 Three-dimensional elastic problem in a shell domain 


In this section we consider a shell domain QS, assumed with constant thickness and 
described from a reference surface X. In what follows, that reference surface will be 
identified to the shell middle surface parameterized by the coordinates &, 7, that is, 
X(é, n), where 


X, n) 
Xn =| X(gn) |. (3.81) 
X3(€,7) 


With n being the unit vector normal to the middle surface, the shell domain as 
can be parameterized from 


x(&,7,0) = X(,n) +O -m. (3.82) 


The geometrical transformation (č, n, Č) > (X1, X2, X3), at its inverse, can be easily 
obtained and expressed into a separated form. 

The weak form of the elastic problem defined in the shell domain 05 using again 
the Voigt notation reads 


| e(u")’ -K- e(u) dx = | u“ -f4 dx + | u“ - F, dx. (3.83) 
0s as Tk, 
Now we are considering the coordinate transformation introduced in the previous 
section mapping x € QS into (f,7,¢) € E = Q x TZ, with (&,n) «Qc Rand(eZIcR. 
The geometric transformation requires to transform the differential operator as 
well as the different volume and surface elements, from which the standard procedure 
applies for computing the separated form of all the kinematics and static variables. 
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3.4.5 Squeeze flow in composite laminates 


The in-plane-out-of-plane separated representation allows the solution of full three- 
dimensional flow models defined in plate geometries with a computational com- 
plexity characteristic of two-dimensional simulations. In the present case the three- 
dimensional velocity field reads 


N 
v(x,z) = © P;(x) © T,(z). (3.84) 
i=1 


The Stokes flow model is defined in = = Q x Z, Q c R? andZ c R, and for an 
incompressible fluid, in the absence of inertia and mass terms it reduces to 


V-o=0, 
o = -pl + 2nD, (3.85) 
V-v=0, 


where ø is the Cauchy stress tensor, I is the unit tensor, 7 is the fluid viscosity, p is the 
pressure (Lagrange multiplier associated with the incompressibility constraint), and 
the rate of strain tensor D is defined as 


_ Vt (vv)? 
es eae 

When considering a laminate composed of P layers in which each layer involves a 
linear and isotropic viscous fluid of viscosity n;, the extended Stokes flow problem in 
its weak form involves the dependence of the viscosity along the thickness direction. 

If H is the total laminate thickness, and assuming for the sake of simplicity and 
without loss of generality that all plies have the same thickness h, it results that h = A 
Now, from the characteristic function of each ply x;(z), i = 1,...,P, 


D (3.86) 


f1 if@-Dh<z<ih, 
Xi@) = | O elsewehere, 3.87) 
the viscosity reads 
P 
n(xz) = > ni X), (3.88) 


i=1 
where it is assumed, again without loss of generality, that the viscosity does not evolve 
in the plane, i. e., n;(X) = nj. 
The Stokes model can be easily extended to power law fluids where the extra- 
stress tensor reads 


T = 2KD%,'D, (3.89) 


with D,, being the equivalent strain rate, as well as to more complex constitutive mod- 
els as the ones involved in composite manufacturing processes. 
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3.4.6 Electromagnetic models in composite laminates 


Conventional processing methods for producing polymer composite parts usually 
involve the application of heat to the material by convection or conductive heating 
through elements, which depend on surface heat transfer. Microwave (MW) technol- 
ogy relies on volumetric heating, that means thermal energy is transferred through 
electromagnetic fields to materials that can absorb it at specific frequencies. Volumet- 
ric heating enables better process temperature control and less overall energy use, 
which can result in shorter processing cycles. Furthermore, comparable mechanical 
properties are shown between parts made with the MW technology and parts made 
with a traditional curing system. These virtues of the MW technology have attracted 
interest in developing the method and adopting it for the production of thermoset as 
well as thermoplastic composite materials. 

The double-curl formulation is derived from the Maxwell equations in the fre- 
quency space, which in the absence of current density in the laminate reads 


vx (iv x) - Wek =0, inQcR (3.90) 
with the complex permittivity e given by 
eee a (3.91) 
w 


and where p, €,, and o represent the usual magnetic permeability, the electric permit- 
tivity, and the conductivity, respectively. 

The previous equation is complemented with adequate boundary conditions. 
Without loss of generality we are assuming in what follows Dirichlet boundary condi- 
tions in the whole domain boundary 0Q, 


nxE= E, in 39, (3.92) 


where n refers to the unit outwards vector defined on the domain boundary. In the pre- 
vious expressions E, is the prescribed electric field (assumed known) on the domain 
boundary, tangent to the boundary as equation (3.92) expresses. 

The weighted residual weak form is obtained by multiplying (3.90) by the test 
function E* (in fact by its conjugate, E`, to define properly scalar products being the 
complex-valued electric field, i. e., E = E, + iE;), and then introducing a stabilization 
to enforce the Gauss law, 


[5.00 xB) (xE )dx-w? | eB-E dx 
à a 


+ | sail” (B) (V. (€E")) dx 
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a aa - (€E)) (V- (EE )) dx J aa . (eE) (n-(€E )))dx=0, (3.93) 


where T is the regularization coefficient. 

To ensure a high enough resolution of the electric field along the component thick- 
ness to represent the multilayered structure, we consider an in-plane-out-of-plane 
separated representation 


N 
E(x, y,Z) = X Py) o T;(2), 


i=1 


where “o” refers to the Hadamard product, and use the standard rank-one update con- 
structor. 


3.5 Conclusions 


This chapter revisited the state of the art and the recent developments in the use of PGD 
for addressing engineering problems. In particular it addressed the pioneering works 
considering space-time separated representations, which were then extended for solv- 
ing multidimensional models encountered in kinetic theory descriptions of complex 
fluids, quantum chemistry, etc. They were also considered for describing and solving 
stochastic models and any kind of parameterized partial differential equations whose 
solutions result in a sort of virtual chart or computational vademecum. These para- 
metric solutions have been successfully employed with multiple purposes: simula- 
tion, optimization, inverse analysis, uncertainty propagation, and control, all of them 
under the stringent constraint of real-time feedbacks. Finally separated representa- 
tions were extended for separating space and efficiently addressing the solution of 
problems in degenerated domains, as for example problems defined in plates, shells, 
laminates, etc. 
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Yvon Maday and Anthony T. Patera 
4 Reduced basis methods 


Abstract: In this chapter we describe the reduced basis (RB) method for parameterized 
partial differential equations (PDEs). We first describe the motivation for RB methods 
in the many-query and real-time contexts and the associated offline-online computa- 
tional paradigm. We next introduce the framework for parameterized PDEs and the 
associated theoretical rationale for reduction. We then turn to projection techniques: 
formulation, a priori and a posteriori error estimation, and offline-online computa- 
tional strategies. We next discuss techniques for identification of optimal approxi- 
mation spaces, in particular the weak greedy approach. We emphasize linear elliptic 
PDEs, but we also consider nonlinear elliptic PDEs as well as linear parabolic PDEs. 


Keywords: weak greedy sampling, empirical interpolation method, Galerkin projec- 
tion, a posteriori error estimation, offline-online procedure 


MSC 2010: 65M60, 65N30 


4.1 Motivation 


Parameterized partial differential equations (PDEs) are important in many scien- 
tific and engineering applications. The parameters typically characterize the spatial 
domain, the boundary conditions and initial conditions and sources, and the coeffi- 
cients associated with the underlying constitutive relations. In general the solution 
of our (say, elliptic) PDE shall be a parameterized field: For given parameter value 
H = (Up ---» Hp) € P, ul(H) € V; here P € R” is a compact parameter domain, and V is 
the appropriate function space associated with our PDE. In what follows, we assume 
that V is a Hilbert space. In the forward context we prescribe u to deduce u(y); in the 
inverse context, such as parameter estimation, classification, and optimization, we 
deduce p from functionals applied to u(y). 

We introduce the parametric manifold M = {u(p) | y € P}. The premise for param- 
eterized model order reduction is well established [28, 1]: For the approximation of the 
solution u(y) for many u € P, we need not necessarily consider a finite element (FE) 
approximation space V, c V which can well represent any function in V; we need only 
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consider a reduced basis (RB) approximation space Vy c Vp which can well represent 

any function in M c V in the sense that dist(M, Vy) is small. We may thus antici- 

pate that the dimension of Vy, N, will be much smaller than the dimension of V}, Np, 
with attendant reductions in computational cost: A query to the RB approximation, 

H € P |> uy(H) € Vy, will be much less expensive than a query to the FE approxi- 

mation, uw € P > up(Ņ) € V,. In actual fact, not all manifolds are “reducible,” where 

reducible here is defined as a sufficiently rapid decrease of the Kolmogorov N-width 

[30, 23]; we shall provide some a priori and a posteriori tests to confirm the latter. Fur- 

thermore, given a reducible manifold, the identification of a good RB space Vy — such 

that dist(M, Vy) also decreases rapidly with N — requires considerable computational 
effort, and in particular many appeals to the FE approximation. 

Parameterized model order reduction thus proceeds in two stages: in the offline 
stage, given our parameter domain P and parameterized PDE, we construct a se- 
quence of parameter-independent spaces {Vjy}-1,..,n,,,,3 in the online stage, for given 
N, we query the RB approximation, u € P +> uy(p) € Vy. This offline-online paradigm 
is computationally relevant if (i) the offline effort to construct Vy can be justified either 
by a real-time or many-query context, and (ii) the online effort to evaluate y > uy(H) 
is indeed much less than the online effort to evaluate u +> u,(). We elaborate on (i) 
and (ii). 

(i) In the real-time context, we simply choose to “write off” the offline effort given 
the stated premium on rapid response in the online (deployed) stage. In the 
many-query context, we explicitly amortize the offline effort over many online RB 
queries. 

Gi) We will typically require not only Nmax < Np but also special structural properties 
of the parameterized PDE and associated solution procedures: This special struc- 
ture is often realized through an empirical interpolation method (EIM) [5] which 
introduces additional “variational-crime” errors; we shall denote the resulting FE 
and RB approximations by u,(w) € Vp and ŭy (4) € Vy, respectively. 


Note from an applications perspective we proceed not from model order reduction to 
context, but from context to model order reduction: A real-time or many-query appli- 
cation justifies an offline-online computational strategy which in turn can be realized 
through (among other strategies) model order reduction. 

We briefly discuss the choice of the parameter dimensionality, p, and parameter 
domain, P. We may consider as a first proposal all parameters of possible interest, po 
large, and a parameter domain Pp which contains all values of uy for which our PDE 
is well-posed (in the sense to be described below). However, in the context of model 
order reduction, the offline and online computational cost will depend on the number 
of parameters and also the extent and “shape” of the parameter domain, and hence we 
must typically accept p < py and hence P c Po: the parameterization and parameter 
domain must be chosen to anticipate the parameter values of ultimate interest in the 
online applications to be considered. 
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We provide a roadmap of the chapter. In general, we emphasize general (second- 
order) linear elliptic PDEs, but we also consider a nonlinear elliptic PDE as well as gen- 
eral linear parabolic PDEs; extension to nonlinear parabolic PDEs is then immediate. 
In Section 4.2 we formulate our parameterized PDEs and summarize the associated 
theoretical foundation for dimension reduction: conditions, or at least guidelines, un- 
der which a manifold is reducible. In Section 4.3 we develop the projection method, 
in fact simple Galerkin projection, by which we determine i,y(y) € Vy; we also pro- 
vide a priori error estimates and a posteriori error estimators, and we describe the 
associated offline-online computational procedures. In Section 4.4 we describe meth- 
ods for construction of the RB approximation space, Vy, with emphasis on the weak 
greedy procedure: The weak greedy procedure efficiently identifies a parameter sam- 


ret 


seep: 


M; the weak greedy procedure, and in particular the rate of decrease of dist(M, Vy) 
with N, is a constructive test of reducibility. 

The prerequisite for this chapter is experience in the formulation, elementary the- 
ory, and implementation of FE methods for PDEs, as well as some exposure to associ- 
ated functional analysis. The intended audience is graduate students and profession- 
als who wish to consider RB methods in their research or design efforts. The chapter 
emphasizes (i) the conditions and hypotheses under which RB methods may prove 
fruitful, (ii) the fundamental ingredients and procedures which must be incorporated 
in any RB formulation, and (iii) the underlying error analysis, a priori and a posteriori, 
which informs successful RB practice. We focus on the “inputs” — related to the partic- 
ular PDE, parameter domain, and context — which must be provided by the prospec- 
tive user, and on methods which can be generally and easily implemented given a 
standard FE foundation. 

Finally, for readers who seek further details, a broader range of alternative tech- 
niques, more general classes of problems, and deeper coverage of both theory and 
implementation, we recommend two recent research monographs on RB methods [21, 
32]. We hope our chapter here can serve as a portal to further study. 


4.2 Parameterized PDEs 


In Section 4.2.1 and Section 4.2.2 we consider linear elliptic PDEs. In Section 4.2.3 we 
consider parabolic PDEs as well as nonlinear elliptic PDEs. 
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4.2.1 Weak form 


4.2.1.1 Formulation 


We introduce a spatial domain Q c RÊ with boundary 0Q; we denote a point in Q 
as X = (X,,...,Xq). We then define the space V as V = {v € HQ) | Vir, = O} for Tp 
a nonempty portion of the boundary ðQ; we denote the inner product and induced 
norm associated to V as (.,-), and ||- ||, respectively. Unless otherwise noted, we shall 
take for our inner product 


(w,v)y = | Vw : VV + Cr WV, (4.1) 
Q 
for c;2 a nonnegative real number. We further introduce the dual space to V, V', of 


linear functionals continuous with respect to || - ||y; we equip V’ with the usual dual 
norm, 


v 
lgly = sup Ig uy Vg eV’. (4.2) 


vev IlVlly 


We also define the Riesz representation of any g in V’, Rg € V, by 
(Rg v) =glv), We. (4.3) 
Finally, we recall that 


Isly =IRelly, Yg € V’, (44) 


which follows directly from (4.2), (4.3), and the Cauchy—Schwarz inequality. 
We now introduce the parameterized linear forms y € P + f(; p) and y € P > 
€(-;). In fact, for simplicity of exposition, we shall assume that 


f~) = [oew + | fog (sm v, €(V3H) = | fasmv + [irem v, (45) 
Q Q 


N,R Iy sR 


where fo(u) € L7(Q), lo(u) € LQ), fiyr) € L’ Tyr), and Cry pH) € L’(Ty.p). Here 
Tyr = 0Q\ Tp is the portion of the boundary on which non-Dirichlet (Neumann or 
Robin) boundary conditions are applied. It follows from our assumptions that f (; p) 
and ¢(-;) are continuous for all u € P. 

We further introduce the parameterized bilinear form u € P > a(;,; 4) : V x V > 
R. In general, a may take the form 


a Ow ðv 


aww = Y [rew (4.6) 


— OX; ax 
ij=0 Q 
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for Y;j € L® (P; L” (Q)) and (for convenience) 0./dx, = Id (the identity operator); note 
that a can also include a contribution such as fona Y(; Hp) w with Y € L” (P; L® (Ty p))- 
We shall need several constants to characterize ¢ our bilinear form a: For any p € P, 
the coercivity constant, a(u), continuity constant, y(u), and inf-sup constant, B(), 
are given respectively by 


T slaw lacw, vs) p law vB 
= inf ————, = pce Bioanal See 
Be Wye. eee wie Pee ee Wl 
(4.7) 


we denote by a = min,<p a(H), y = MaX,<p Y(H), and $ = min,.p (H) the correspond- 
ing worst case quantities over the entire parameter domain. For economy of presenta- 
tion we define 


a( for the coercive case, 
Su) = l p) (4.8) 


BQ) for the noncoercive case; 


we may also write c°(u) = max(a(j), B()), however for computational purposes we 
prefer the more explicit definition (4.8). We can then state our hypotheses on a: in the 
coercive case, a is positive and y is finite, and in the noncoercive case, $ is positive and 
y is finite; more succinctly, we require cê positive and y finite. 

We now define the weak form of our parameterized PDE: Given p € P, find a (or 
the) field u(y) € V such that 


a(ul), v;p) =f(vViw), Wwe, (4.9) 


and evaluate the scalar output s(w) € Ras s(u) = £(u(p); p) (We implicitly assume 
that all inhomogeneous essential boundary conditions up(u) € H 1/ ?(Tp) are lifted and 
hence implicitly incorporated in f(-; y).) It follows from our hypotheses on a and f and 
the Lions—Lax—Milgram—BabuSka theorem that (4.9) admits a unique solution for all 
H € P; furthermore, from our hypothesis on £, s(j) is finite for all y € P. (In fact, 
for the noncoercive case, we require a third condition on a, typically satisfied in our 
context.) 

We provide a simple illustration, which we denote Example 1.0. We consider a 
connected open domain Q c R*? and further assume that Q is decomposed as the 
union of two nonoverlapping open subdomains, Qg) and Q: Q = QW) U QW) and 
Qa) N Qg = 9. We set p = 3 and introduce parameter y = (p1, H2, 43) € P c {v € R? | 
vı > 0, V2 > 0}. We then define 


a(w,v; 4) = | yyVw- Vv + | Vw: Vv + | nw, Vw,v eV’, (4.10) 
Qo Qo Q 


1 We may associate to our output functional a dual problem and corresponding adjoint; the latter can 
serve (for example) in the development of improved RB output approximation and error estimation 
[25]. In the interest of space, we consider only simpler primal-only approximation. 
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as well as 


fusp = fa +4;3X%ı)v and £(v;u)= | v. (4.11) 
Q INR 


We can readily demonstrate that a is continuous and coercive, and furthermore sym- 
metric, and that f and £ are continuous. (We can also include a convection term: For 
a divergence-free convection velocity which is furthermore outward on Ty g, the bilin- 
ear form a remains continuous and coercive but will no longer be symmetric.) In the 
case in which Q is polyhedral we would expect u(y) € H’ es for Oregularity > 0; the 
latter would limit the convergence rate of the FE approximation, but need not limit the 
convergence rate of the RB approximation. 

We comment briefly on the treatment of parameter-dependent geometry. In our 
exposition we shall consider only the case in which Q is independent of u. In actual 
practice, we may treat problems in parameter-dependent geometry, y € P +> Qorig(H). 
In that case we introduce V,,,.(@) = H orig (H)) and bilinear and linear form u € P œ> 
Aorig(s 5 H) : Vorig(H) x Vorig) > Randy € P > forigls H) : VorigH) > R, respectively; 
we then seek Uorig(H) € Vorig (H) solution Of dorig (Uorig (H); V; H) = forig V; H), WV € Vorig(H). 
RB methods will rely on some similarity of solutions on the parametric manifold: We 
thus map Qorig to a parameter-independent reference domain Q, y € P > T(;p) : 
Q > Qorig(H); a variety of RB-relevant mapping procedures are described in Chapter 1 
ofthis volume (Volume II) of this handbook. We thereby arrive at the statement (4.9) for 
U(H) = Uprig(s H) ° T (5M), which is then the point of departure for the RB formulation. 
Note for the case of parameter-dependent geometry the functions Y;j(-;1),0 < i,j < d, 
of (4.6) will include the usual transformation terms associated with the Jacobian of 
the mapping function 7(-; y). 


4.2.1.2 FE approximation 


In general, it, (jz) approximates u(p), but typically we must construct the RB approxi- 
mation through a computable intermediary or “surrogate”; in the context of this chap- 
ter, and quite often in practice, the latter takes the form of an underlying finite element 
(FE) approximation. Towards that end, we introduce a conforming FE space V, c V 
of dimension N,. We shall choose V, such that, for any u € P (say), uu) - u,(W)lly < 
tol, /2, where toly is the prescribed error tolerance; we will then subsequently require 
\|u), (4) — tty (Wlly < toly/2 to ensure |lu(u) - üy (Wlly < toly. 

We shall require for purposes of our subsequent RB approximation that V, is inde- 
pendent of u. As we shall see, the RB online cost is largely independent of N;,, however 
the offline cost will indeed depend on N,, and hence we may be conservative but not 
profligate in the design of the FE approximation space. We might construct V, as fol- 
lows: Consider a representative sequence of parameter values Ep, = {pir E Pẹi=1,. Ke? 
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initialize the FE approximation for parameter value piss vi, as vi- (and for vi choose 
an initial uniform coarse mesh) and adaptively refine to the desired error tolerance 
toly/2; set V, = vý =, We further introduce the dual space to V}, Vi. of linear func- 
tionals continuous with respect to || - |y for all functions in Vp; we equip V, with dual 
norm 


Iall; = sup E, (4.12) 
vev, Illy, 
We may then define the Riesz representation of any gp in V}, Ragna € Vn, by 
(Raw V)v =8n(v), YV € Vp (4.13) 
in terms of which we can evaluate the dual norm from 
lSaliv; = VRa8nllv- (4.14) 


We note that (4.13) is a finite-dimensional problem. 
We now define the (continuous) Galerkin-FE approximation: Given u € P, find 
u,(H) € Vp such that 


a(up(H), v3) =f), YW € Vp, (4.15) 


and evaluate the scalar output s,(m@) € R as s,(u) = (u (H); p). We denote the FE 
version of the constants in (4.7) - with V replaced by V, — by subscript h: for all 
H € P, au) < aa(H) < Ya) < yu); however, in general f,(u) > p(y), and thus 
in the noncoercive case we include the additional hypothesis (on V,,) 6}() > O, in 
order to ensure well-posedness of the FE approximation. We may then also define 
the corresponding worst case constants (over P) as dp, Yp, and $p. Finally, we intro- 
duce Ch (u) = max(a,(p), B,(M)) and corresponding worst case (minimum over P) con- 
stant cp. 

We next represent V; by a nodal basis (phy associated to nodes pes € 


a N3 given any function v} € Vp, we shall denote by v, € R™ the correspond- 
ing vector of (nodal) basis coefficients. The FE discrete equations now directly follow 
from (4.15) and our basis for V,: Given y € P, find u, (y) € IR™ such that 


Apu, (H) = fa), (4.16) 


and evaluate the scalar output s,(u) € R as s,(u) = ewu, (p) (for T the transpose 
operator). Here Aj,(u) € R™ , fp(u) € R™, and £,(u) € R™ are given by 


(AnD) =al phu) ED) fl) (ED); =el) 1<ijs Ny 
(4.17) 
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note that A, and f, are the FE stiffness and load vector, respectively. 
For future reference we also introduce the parameter-independent inner-product 
matrix X, € RAMa, 


(Xaj = (py 1<ij< Nhy (4.18) 


We note that for any w} € Vh, Yn € Vp, (Wa Vay = w,XnVn- Furthermore, it follows 
from (4.13) that for any g, € V} the FE basis coefficients of the Riesz representation 
Rp are given by X;,'g), for (Sn): = g(t), 1 < i < Nj. It follows from (4.14) that the 
dual norm ||g,ll v, may be evaluated as 


ae (4.19) 


(S,Xn Br 
note that we require only the action of X;', which might be effected (in the direct con- 
text) through Cholesky decomposition and subsequent forward/back substitution. 

In actual practice the FE matrices and vectors are formed by numerical quadra- 
ture: The integral of (4.6) is replaced by a corresponding sum with quadrature points 
and weights 

anap co, pre eR. j=... hia (4.20) 
and the integrals of (4.5) are replaced by corresponding sums with quadrature points 
and weights 


Peira pp eRe. Jaina e, (4.21) 


for simplicity of exposition, we shall presume that the error induced by quadrature is 
negligible relative to toly. The latter is plausible if h is sufficiently small and further- 
more Y; O <ij< d, fo, fiyr £o, and Cyr are sufficiently smooth. Note that in practice 
the FE quadrature is effected as a sum of elemental quadratures. 


4.2.1.3 Affine parameter dependence 


Affine dependence of the forms {a,f, £} on the parameter u greatly reduces the com- 
putational complexity of the online stage of the RB method. We note that parameter- 
ized model order reduction can proceed without (appeal to) affine parameter depen- 
dence - but less effectively than if we can and do take advantage of affine parameter 
dependence. We thus wish either to confirm affine parameter dependence or alterna- 
tively, and more generally, to impose affine parameter dependence though approxi- 
mate forms {4@, f ,¢} = {a, f, €}; we pursue here the latter, which includes the former as 
a special case. 
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In particular, we introduce a, f, and # which can be expressed, for all u € P, as 


Qa ei Q z Qe 
lw, v;p) = X 03a (wv), fiw) =) fv), Eww = Y OF eV), 
q=1 q=1 q=1 
(4.22) 


where 04 : P > R, 1 < q < Qa, O} : P > R, 1 < q < Qf, and O% : P > R, 1 < q < Qe, 
are suitably smooth functions, af : VxV —> R,1 < q < Qa, are parameter-independent 
continuous bilinear forms, and f1 : V > R, 1 < q < Qf, £1 : V > R, 1 < q < Qe, are 
parameter-independent continuous linear forms; we assume the forms are linearly 
independent. 

Given u € P, we now seek ii(y) such that 


ă(ŭ(u), v; p) =f(viw, Wwe, (4.23) 


and evaluate the scalar output §(w) € R as š(u) = č(ŭ(y); p). We also define the corre- 
sponding FE approximation: Find u,(y) € V, such that 


Q(i,(H), v; H) =f(v;p), Yv € Vp (4.24) 


and evaluate the scalar output 5„(4) € R as §,(u) = @(i,(M);m). We denote the FE 
stability and continuity constants (hence over V,) associated to @ by &„ (4), 7,(w), and 
Baj) for any u € P; we may then also define the corresponding worst case constants 
(over P) as n, Ÿn, and Br- Finally, we introduce Cp (HM) = max(a@(y), Br) and the 
corresponding worst case (minimum over P) constant ¢;. We shall shortly provide a 
perturbation result for č}. 

The discrete FE equations now read 


A, (With = fa) (4.25) 


and §),(#) = Dü), where À (u) € IRN Nn, f (u) € R™, and @,(u) € R™ are given 
by 


(Anw) äl pu) Fr) Fl u) WD) = Elou) 15i) s Np 


(4.26) 
We further note from (4.22) and (4.26) that 
7 Qa d Q; 5 Qe 
An) = ) OFA; fn) = Y Gf Er =} Oe (4.27) 
q=1 q=1 q=1 
for parameter-independent A} ce RMN 1 <q < Qa fi € R™,1 < Qr, and 


<q 
ele RX, 1< q < Qg; for example, (Af) = alg, p), 1< i,j < Np1<q< Qa 
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We must now seek affine approximations (4.22) such that {@,f, 2} is sufficiently 
close to {a,f, £}, and hence ŭ and i, are sufficiently close to u and u,, respectively. 
Towards that end, we first introduce (a restricted form of) the EIM [5]. We are given 
integer Nyy = 1 (in practice, large) and a function g : P > RNs; we further define 
a train parameter sample of size Kay, Eem = {Hi € Phizt,...Kem(C P) and generate 
the associated snapshot set Grm = {8() }ucs,,,- Lastly, we prescribe norm || - |g and 
associated error tolerance tolp. 

- Inthe offline stage, we execute Algorithm 4.1 (presented in detail below) to obtain 
interpolation indices {i} € {1,...,Nem}m=1,..m> associated interpolation vectors 
{é™ e RNemy met,..M> and a nonsingular lower triangular interpolation matrix By € 
RY xM y 


— Inthe online stage, given u € P, we approximate g(p) as 
M 
ED = Y} bm g", (4.28) 
m=1 


where b(u) € R“ is the solution of Bub(p) = g*(p) for g* (p) € R“ given by 
(g°(W))m = (EW); 1 <m < M. For succinctness in the description of Algo- 
rithm 4.1 we define Zy : Rim — REM such that (4.28) reads gu) = Tug) 
for M > 1; for M = 0 we set Zy = 0. 


Algorithm 4.1: Empirical interpolation method (EIM): The EIM algorithm is of the 
“greedy” variety (also invoked in the identification of Vy, as discussed in Sec- 
tion 4.4.1): on line 4 we choose for the next parameter value the point in Egry for 
which the current EIM approximation is worst; within the EIM context, this strat- 
egy in conjunction with line 5 also ensures a stable interpolation procedure. We 
assume that the set Gryp is not embedded in a small finite-dimensional space. 
Data: Ney Erm g : P > RM (in fact, Germ suffices), || - ller tolem 
Result: M, {i*, € {1,..., Nem }iemem> 1” € RY™ remem» By € RO” 
1 Set M = 0 and err = oo; 
2 while err > tolg do 
3 SetM — M +1; 
4 Find pw" = arg supyes,,, IZOD - Zvu18 (lems 
5 Find iy = arg supjeq1,. Ney} (SH) - Zu 18")) ils 
6 Define” = (g(u") - Zy_18(H")) / (S(H") - Zu ash" i; 
7 Update (Bm); = Fie <j,k <M; 
8 Set err = |Ig(u") - Zy 8 llem: 
9 end 
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By construction, for any u € Egm ZH) - S(H)llem < toleny. However, the EIM ap- 
proximation may (and must, in practice) be applied to values of u € P which do not 
appear in the train parameter sample Zp.” We note that in infinite precision we ob- 
tain M < dim(span{Ggm}). 

We now apply the EIM to develop approximate affine forms for our PDE. We con- 
sider the coefficient function Yoo of (the numerical quadrature version of) (4.6). We 
set Ngm = patago (or yee for fry p Cryg) identify (gH); = Yoon u),1 <i < 
N Anada and choose ||: |lgm = ll Il. We then apply Algorithm 4.1 to obtain M, the inter- 
polation indices {iž}<m<m» interpolation vectors {€”"},<m<y, and interpolation matrix 
By associated to the Yo, contribution to the affine sum for the bilinear form å. (Each 
of the other coefficient functions Y; 0 < i,j < d, and (i,j) + (0, 0), fo, and £o is treated 
[separately] in the same fashion, as well as Frye and Erur .) Note that the correspond- 
ing © (u) correspond to the b.(p) of (4.28) and are thus defined implicitly in terms of 
interpolation indices, vectors, and matrices; the corresponding a` (w, v) are given by 


awed LQ 


2E gi (xe 2) E (oe 2) poe LQ (4.29) 


for quadrature weights and points defined by (4.20). 

Assuming that each of the coefficient functions resides on a low-dimensional 
parametric manifold, we may anticipate that we can obtain a corresponding EIM ap- 
proximation with relatively few terms, M small. For any given D c P, we define the 
error induced by the EIM approximation in our bilinear and linear forms as Eos 


Gok = sup max sup |a(w, v; u) — a(w, Wel or If(v; w) -fvw 
yueD weV,veV Iwllyllvlly vey Ivy 
veV Wal 


For a of the form (4.6) application of the Cauchy—Schwarz inequality yields etem asa 
function of tolp; for example, in the absence of off-diagonal terms in (4.6), con = 
max(1/c;2, 1) tolzyy. We can further argue that, for Egyp sufficiently rich, ae = ent 
an adaptive procedure has been proposed in [27] to support this argument. Finally, it 
can readily be demonstrated that 


Cee. e (4.31) 


2 Ina similar fashion, in the most general EIM formulation, g(u) corresponds to the evaluation of 
a function g : Q x P — Rata set of (here, quadrature) points in Q; however, the resulting EIM 
approximation can then be applied for any x € Q. 
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and also 7, < Yh + a it follows that our EIM-perturbed FE problem is well-posed for 
efm sufficiently small. 

There is an alternative EIM approach to the development of the affine forms: the 
operator EIM (OEIM) [15]. In this case we would apply the EIM method directly (for 
Nem = Np) to fh), €a (H) and (for Nem = Nz) to Ay, € R; the latter is the single- 
index (vector) form of the stiffness matrix Ap, which is then repacked in double-index 
form once the OEIM is complete. The OEIM has the important advantage of nonintru- 
siveness: the affine approximation may be deduced solely from the FE stiffness matrix 
and load vector without any knowledge of the associated formation processes, thus 
permitting a general interface between FE code and the RB code. It is important to 
choose a norm ||- |g for the OEIM procedure to permit ultimate error control of the 
RB approximation in || - |y. For fn and £, we may choose ||ôgllem = (6g' X;,'6g)"/ oe 
which can be reasonably rapidly estimated. For A), the relevant norm can be evalu- 
ated as the square root of the maximum eigenvalue jean associated to the generalized 
SPD eigenproblem ôA; X; 6A), yee = APP, oan , hence somewhat cumbersome. For 
these norm choices we directly obtain en = tolg. In summary, the OEIM offers a 
very easily implemented procedure for the construction of affine approximations. 

We briefly revisit Example 1.0, described by equations (4.9)—(4.11). We first note 
from inspection that we can directly choose {a, f, 2} = {a,f, €} to obtain an affine rep- 
resentation with Q, = 3, Qf = 2, Q; = 1. For our particular example Yoo, Yy, Y22, and 
Y33 are nonzero, and hence the EIM procedure described — which treats each term in 
the expansion (4.6) separately — would yield Q, = 7; a concatenated EIM — in which 
we treat all the Yy~Osijs d, within a single EIM — would recover Q, = 3. The OEIM 
procedure, which treats the entire form, would directly recover Q4 = 3, Qf = 2, Qe = 1. 
It is often the case for problems in which the geometry does not depend on the param- 
eter that {a,f, £} admits an exact affine representation, Ce = 0, with relatively few 
terms. However, in the presence of parameter-dependent geometry, and in particular 
nonaffine geometry transformations, {a,f, 2} will not admit an exact affine represen- 
tation. 


4.2.2 Justification for reduction 


The fundamental hypothesis made on the parametric manifold M = {u(y) | u € P} 
introduced in the first section is its “reducibility” in the sense that there supposedly 
exist(s) some (series of) finite-dimensional space(s) Vy that approximate well M in 
the sense that, denoting by dist(M, Vy) the deviation of M from Vy, i.e., 


dist(M, Vy) = sup inf lu) -vylly, 
u(H) VyEVy 


dist(M, Vy) is decreasing fast with N increasing. The question we want to raise here is: 
Why should it be so? And also, what is that (series of) finite-dimensional space(s) Vy? 
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This hypothesis is formally stated by going one step further in the definition of the 
deviation, i. e., introducing the quantity 


dy(M, V) = T Pe dist(M, Vy), (4.32) 
which is known as the Kolmogorov N-width [30, 23] and represents the ability of M to 
be approximated by some optimally chosen vectorial space of dimension N. 

As has already been remarked above, the “optimal” choice Vy depends on M, and 
the optimal choice for M will not be valid for another set of functions. 

This notion is the right one, indeed, when, e. g., dy(M, V) goes to 0, like pọ” with 
O < p < 1or even like cN” with p large enough (say, p > 6), then we are in a good 
shape, expecting that, for any u € P, very few (well-chosen) degrees of freedom (the 
coefficient in some appropriate basis of Vy) will be sufficient to approximate well any 
u(H). 

We know that in some cases, e. g., a linear structure of the PDE, by superposition it 
is possible to check that is finite-dimensional. Of course this is neither the generic 
case nor the case we are interested in. Then typically, regularity of the solutions with 
respect to the spatial variable may lead to propose, for every u, a high-order (say, poly- 
nomial or spectral) approximation that converges rapidly, even exponentially (when 
the degree of the polynomial increases) and thus, Xy can be chosen as the set of poly- 
nomials of degree < cN’/, This will not be the optimal space but the optimal choice is 
better than the polynomial choice and thus the best polynomial fit provides an upper 
bound for the Kolmogorov N-width. However, the fact that polynomials approximate 
well any regular function, regardless of the property of the whole set M, i. e., its struc- 
ture, shape, coherence, makes it understandable that this “generic” choice cannot be 
the optimal one for M, and that it may be much, much better. 

As noted in [12], if the mapping u € P +> u(y) is linear continuous, then the Kol- 
mogorov N-width of M is upper bounded by a constant times the Kolmogorov N-width 
of P. The generalization of this statement that is proposed in [12] is that if the previous 
mapping is holomorphic (meaning that u(y) has a Fréchet derivative at any parameter 
p belonging to a compact set K c P), then, for anys > 1and t < s -1, 


supn'd,(K,P) < co = supn'd,(u(K), V) < oo, (4.33) 
n21 n21 
where u(K) = {u(u),u € K}. The loss of 1 (with respect to the linear case) may be 
not optimal but this result provides extra reasons for dy(M, V) to be small when u(y) 
is the solution to some parameter-dependent PDE. Indeed, it is well known that, by 
differentiating the PDE, V,,u() is the solution to a similar PDE as u(y), and thus under 
a reasonable hypothesis, this set satisfies the holomorphic assumption. 
In order to explain faster rates of convergence, the first analysis we know ofis [26], 
where exponential convergence was proven for a simple one-dimensional parameter 
space elliptic PDE. More recently a general extension was performed in [3], where, by 
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using, in a constructive way, low-rank tensor approximations for families of elliptic 
diffusion PDEs parameterized by the diffusion coefficients, the authors have been able 
to derive exponential convergence rates in a much more general framework. 

A typical case where the Kolmogorov N-width is not small (at least in a straight- 
forward manner) is when the problem is convection-dominated. Indeed, the set of so- 
lutions to a simple pure linear convection problem, when the velocity is among the 
parameters, is the initial solution, properly translated: If it is not regular, then the set 
of all solutions is of very large Kolmogorov N-width. There are ways to circumvent this 
(see, e. g., [10, 7]) but it is out of the scope of this contribution. Another case where, a 
priori, the Kolmogorov N-width is not small (but it is simple to fix) is for elliptic prob- 
lems where the right-hand side contains pointwise singularities, the position of which 
may vary and is one parameter of the problem. The “trace” of these singularities of the 
right-hand side can be clearly “seen” on the solution u(y) itself and this may lead also 
to a large Kolmogorov N-width. A simple postprocessing of the solution’s manifold 
through a change of variable that maps the singularities’ positions to a fixed refer- 
ence position allows to better compare the solutions and check that indeed, when the 
singularities are sort of “aligned,” the set of all (postprocessed) solutions is of small 
Kolmogorov N-width. 


4.2.3 Extensions 


4.2.3.1 Generalization of linear elliptic problems 


The extension of our parameterized PDE formulation to vector fields is very simple. In 
particular, and if we assume that Dirichlet conditions at any point on the boundary 
are always applied to all components of the vector, we need only redefine vi > V. 
Thus linear elasticity readily falls into the framework considered in this chapter. It is 
interesting to emphasize here that the manifold of all vector fields (solutions to the 
problem of interest when the parameter varies) can be considered as a whole, which 
leads us to approximate a vector solution as a linear combination of vector (reduced) 
basis functions with scalar coefficients, thereby further decreasing the complexity of 
the online procedure. Note that we shall not consider here saddle problems: These 
mixed formulations — such as the incompressible Stokes equations or the equations of 
linear elasticity for Poisson ratio approaching 1/2 — require special RB treatment [34]. 

The extension of our parameterized PDE formulation to complex fields — “com- 
plexification” for short — is also relatively simple. We must change R to C, interpret |- | 
as complex modulus, consider spaces V of complex-valued functions, conjugate the 
argument of all linear (now anti-linear) forms and the second argument of all inner 
products and bilinear (now sesquilinear) forms, and in our discrete representations 
replace transpose ' with Hermitian ". We provide a simple illustration, the Helmholtz 
problem of acoustics, which we shall denote Example 2.0. 
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We consider Q c R43. We set p = 2 and introduce parameter u = (Mp2) € PC 
{v € R? | vı > 0,V> > 0}. We then define 


a(w, Vv; H) = fa +im)Vw-Vř-umw7, Yw,ve€V?, (4.34) 
Q 
as well as suitable f (; u) and £(-; u). Here i? = —1 and 7 denotes the complex conju- 


gate of v. For u > O (positive dissipation), a is coercive. For yy = 0 (no dissipation), a 
is inf-sup stable unless y, is an eigenvalue A of the associated “resonance” problem: 
Find (y € V,Aresonance € R) such that fo Vy - VV = Aresonance Jy XV» WV € V, where here 
V is our standard real space. In some applications, such as the elastodynamics exten- 
sion of (4.34), the actual dissipation can be substantial; in other applications, such as 
acoustics, 4, must often be interpreted as a numerical regularization parameter. 


4.2.3.2 Evolution problems: parabolic PDEs 


We shall consider here only parabolic PDEs. In fact, RB methods can also readily be 
applied to hyperbolic PDEs, for example the second-order wave equation, however 
in the absence of adequate dissipation the treatment of rough initial conditions and 
limited regularity remains an outstanding issue as indicated above. We shall assume 
for our parabolic PDEs that our bilinear form a is coercive; the noncoercive case is 
more difficult in particular as regards effective a posteriori error estimation [36]. 

We introduce the time variable t and temporal domain (0, T], and the space LQ) 
and associated inner product (-,-)ọ and induced norm || - llo. We further define inner 
product y € P > m(.,;p) : Vx V > R which induces norm m"?(.,; u) equivalent 
to || - lọ. We now state the weak form of our parabolic PDE: Given u € P, we look for 
u(u) € C°((0, T]; L?(Q)) NL7((0, T]; V) such that, for any time t, 


m( A 


T vin) + a(u(t; W), v; p) = Tf (v;y), We, (4.35) 


where t € L?°((0, T]) and f(;4) € L?(Q). We take for initial condition u(t = 0; 4) = 
uo € L*(Q); in actual practice, we may also permit parameter-dependent initial condi- 
tions. Note that we shall not explicitly present the treatment of the linear functional 
output, as (in the absence ofan adjoint) the latter differs little between the elliptic and 
parabolic cases. 

We inherit the underlying FE approximation from the elliptic problem of Sec- 
tion 4.2.1.2. We further assume that m admits an (EIM-approximate) affine expansion, 


Qn 
(w, vs) = > OF, quym"(w, v), (4.36) 
q=1 
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for 07, : P > Rand parameter-independent m’ : V x V > R,1<1< Qm. We may 
then further define our FE mass matrix, Mp (m) € RNM, 


(MD) = A" OSB), 1< k,n < Ny (4.37) 
which we may form as 
Qn 


M,() = > 07, qoM#, (4.38) 
q=1 


for M? = mp", p*), 1 < kn < Ny,1 < q < Qm. We also introduce a “truth” finite 
difference discretization in time: we choose At = T/J and define t = jAt, O <j <J. 

We can now state the FE approximation: Given y € P, we look for id, . (= 
u(t, 34W) € V;,j =1,...,J, such that 


i 0 a | i. 
(na sn) + A(t), aV; H) =qT(t)fv;p), Yv € Vy; (4.39) 


we impose the initial condition TA (p) = Up. We can then state the discrete equations 


to be solved at each time t: 
x Ti- 4 1. a ne 
(anw + E Ma) ) it, ae) = Ar Ma ee PD + T(P fhd. (4.40) 


Although we consider here Euler backward temporal treatment, the methodology 
readily extends to higher-order temporal discretizations. 


4.2.3.3 A nonlinear elliptic problem 


Although our presentation of linear elliptic PDEs and also linear parabolic PDEs is 
rather general, the scope of this article does not permit similar treatment of nonlin- 
ear problems. We thus focus on a particular nonlinear elliptic PDE with rather simple 
structure and underlying theory. In the linear part of this chapter our goal is to provide 
a complete picture of the state of the art, albeit with a balance between performance 
and simplicity, and indeed an emphasis on the latter. In this nonlinear thread our goal 
is less ambitious: We highlight the new difficulty introduced by nonlinearity and the 
corresponding new ingredient — hyperreduction — developed to address this difficulty. 
The hyperreduction treatment presented, as well as other hyperreduction approaches 
[35, 17, 40], is broadly applicable. However, the a posteriori error estimator takes ad- 
vantage of our particular simple nonlinearity, and our offline computational proce- 
dure is rather inefficient. 


4 Reduced basis methods =—— 155 


We consider a particular nonlinear elliptic PDE with monotonic nondecreasing 
nonlinearity: Given u € P c {v € R | v > 0} c R?™', find u(y) € V such that 


| vuw -Vv + | nun) v=u | v, Wel. (4.41) 
a a a 


We require that n € C’(IR) and furthermore n(z>) - n(z,) > O for z) > zı. Two exam- 
ples are the classical smooth test problem given by 7(z) = z? and the more relevant 
(and nonpolynomial) drag law n(z) = |z|z. It can be shown that (4.41) admits a unique 
solution. (Note for this nonlinear problem we prefer explicit rather than abstract rep- 
resentation of the weak form.) 

We may then directly introduce the corresponding FE approximation: Given p € 
P, find up(H) € Vp such that 


| vun -Vv + fnan) v=y | v, WeV, (4.42) 
a a a 


In actual practice, the integrals in (4.42) should be interpreted as quadrature sums, 
hence 


=u > pt v(x tao), Ww € Vp. (4.43) 


For future reference and for purposes of consistency with previous notation we define 
for this nonlinear problem a = a(p) = 1and a, = a;,(p) = 1. 

We now prepare an affine version of our nonlinear problem. Towards that end 
we apply the FIM approach, in particular Algorithm 4.1, to g given by (g()); = 
(up xt; w), 1< i < N42, we specify NNE, ENL I- INE; = I- leo, and tolk, 
and denote by Zy : Vp — V, the resulting interpolation operator as characterized by 
M, {intm-1..m 1$ EET and By. (We provide an NL superscript to the inputs, but 
context suffices to indicate the NL for the outputs.) We note that each evaluation of 
g(p) foru € oe is now expensive — solution of the FE approximation to our problem 
— and not simply evaluation of a coefficient function; more efficient alternatives are 


proposed in the literature [13]. Then, given u € P, we look for ii, (uw) € V, such that 
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pwed 


X pee Vit, (x EE DE Vy(xg"a Py 


M pede 
= x d, d, d, 
S CDan Ni ew) 5 (EM) vot) pas e 
m,m'=1 a j=1 


=u > prey v(x N Wv e Vp. (4.44) 


Under the assumption of exact quadrature we may write (4.44) as 


| van -Vv + EACAN v=y | v, Wwe, (4.45) 
a a a 


where Zy is the EIM interpolant operator. We emphasize that Zy is developed through 
Algorithm 4.1 for n(u,, Cage Naani ; #)), but then applied in (4.45) to n(i, Camas “yess ; W9). 
We say that (4.44) is “affine” in the sense that 7(i),(-;@)) no longer appears in the 
quadrature sum associated with the nonlinear term. The computational importance 
of this simplification will become clear when we consider RB projection. 


4.3 Projection 


4.3.1 Elliptic problems 


4.3.1.1 Galerkin projection 


We consider here the real case, but note that our “complexification” transformation 
may be directly applied. We are given a hierarchical set of RB spaces {Vy}y-1,..N...° 
In fact, this section is applicable to any RB spaces, but for purposes of concrete- 
ness we sketch here the particular space we shall propose in Section 4.4.1. We first 
introduce parameter sample Sy = {pl € Phi-t,...Ninax? the Optimal choice of which 
shall be discussed in Section 4.4.1. The space Vy, for any given N, is then defined as 
span{it),(w’),j = 1,...,N}. Note that, since p’,1 < j < Nmax» are independent of N, our 
RB spaces are nested: V; CV) C +++ C VN yax 

We now define the RB-Galerkin approximation for some given N € {1,..., Nmax}: 
Given 4 € P, find ity (H) € Vy such that 


(ity (W, v; 4) =fvsw), W € Vy, (4.46) 


and evaluate the scalar output šŠy(4) € R as šųķ(ŅĄ) = (ùy (W; H). We recall also that 
we shall denote by uy(y) the RB approximation in the absence of EIM errors; the lat- 
ter corresponds to an effectively exact affine expansion such that ef; = 0 and thus 
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{a,f, Ë} = {a, f, £}. We denote the RB stability and continuity constants (hence over Vy) 
associated to @ by &y(H), Yy(H), and y(u) for any u € P; we may then also define 
the corresponding worst case constants (over P) as ay, yy, and By. We also introduce 
cy (u) = max(@y(p), B (4) and corresponding worst case (minimum over P) constant 
Cy- It is readily demonstrated that y > p, however in general By & By. In the nonco- 
ercive case we must therefore incorporate Bn > 0 as an additional hypothesis. Alter- 
natively, we may consider a minimum-residual projection [25], which ensures a stable 
RB approximation, indeed By > B,; however, Galerkin projection — the simplest and 
least expensive — is typically quite effective in practice. 

We also introduce a basis for Vy, {G }n-1,. 11 < N < Nmax. We shall choose, for 
purposes of stability, an orthonormal basis: (G7, 6;")v = Ompl < m,n < N, for mn 
the Kronecker delta symbol. Given any function vy € Vy, we denote by vy € R” the 
corresponding vector of basis coefficients. The discrete RB equations then read 


Ay@)ity®) = fv) (4.47) 
and 

Sy (HW) = By (Wü (p), (4.48) 
where Ay(p) € RY, f y(u) € IRN, and y(u) € R” are given by 
(Ay @)) im = Ala aH) FND)m FaH) (En) mn = Ela) 1sm nN. 


(4.49) 
We further note from (4.22) and (4.49) that 
F Qa e Q; 7 Qe 
Ay =} OwA fuw= >) of@rh, Ëy => obe, (4.50) 
q=1 q=1 q=1 


for parameter-independent Aj, € RYN, 1<q< Qa fire RY,1<q< Qr, and £}, € RY, 
1< q < Qg; for example, (Af) mn = AG". 0n) 1 <m, n<N,1<q< Qa. 

In actual practice we may express our RB matrices and vectors in a nonintrusive 
fashion which invokes only standard operators and operations readily available in 
the FE context. To begin, we introduce, for 1 < N < Nmax» the RB “basis matrix” Vy € 
RPN, (Vy)in = (Çh) 1 <j < Np 1< n <N. It then follows from (4.49) that 


NGD) =VyAnWVy, fvQ)=VyFnQD, ËN = Vyer. (4.51) 


In the same fashion, from (4.50) (or directly (4.27)), we may write 


AZ = VyAIVy, 15q5Q, Fi =Vyft1sq<Q, €l =VREL 1< q< Qe 
(4.52) 


We recall that the FE matrices for our nodal basis are sparse, and hence the operation 
count to form (say) Ah, for Aj, already formed and represented in sparse format, is 
O(N?N,) floating point operations (FLOPs). We discuss operation counts further in 
the context of the offline-online decomposition. 
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4.3.1.2 A priori error estimation 


It is a simple application of Céa’s lemma and Strang’s first lemma to demonstrate [18] 
that, for any H € P, 


> P 
> Yu \ € Fs lly: 
Ian -üy < (1+ FE) inf fant — wal +Sat(1+ BE) casa 


pe OE 
best-approximation error 


(We can then readily develop an associated error bound for Sp — $y; we defer discus- 
sion of the latter to the a posteriori context.) Note in the error estimate (4.53) that up (4) 
is the “true” FE approximation on which we build the RB approximation and with re- 
spect to which we estimate the RB accuracy, and ŭy (4) is the actual RB approximation 
including EIM approximation of the bilinear and linear forms; hence the bound (4.53) 
reflects both RB and EIM contributions to the error. In Section 4.4.1 we shall develop 
estimates for the decay of the best-approximation error with N relative to the corre- 
sponding Kolmogorov N-width associated with our parametric manifold. 

Finally, we note that in the coercive case the RB discrete equations are provably 
well-conditioned under the assumption that ef, < a; the essential ingredient is or- 
thonormalization of the RB basis functions with respect to the V inner product. In 
particular, it is readily demonstrated that, in the coercive case, the condition number 
of Ay(w) (measured in the usual £-norm) is bounded by (y + Con /[(a- efin for all 
H € P and independent of N. 


4.3.1.3 A posteriori error estimation 


4.3.1.3.1 Dual norm of residual 

A posteriori error estimators shall serve to control the error in the RB approximation: 
in the offline stage to (inexpensively) identify good spaces Vy; in the online stage to 
verify any particular query u > iy(H), Sy(H). In principle we would wish to control 
in both the offline stage and the online stage the total error lu(u) - iy(w)|ly, and in 
certain particular (but important) cases this is indeed possible [39]. More generally, 
we write |lU(M) - UyQolly < lugo) - u,Qwlly + lu 0) - tyGQo|ly: In the offline stage 
we control both ||u — up(H)lly (as described in Section 4.2.1.2) and jup (4) - ùy (Wlly 
over respective (finite-cardinality) train subsets of the parameter domain P, Epp and 
Ep: =rp; in the online stage, for any y € P, we control - and in particular verify 
— only |lu, (Ņ) - iy (ly. We justify the online emphasis on only the FE-RB error: the 
operation count for the online stage shall not depend explicitly on N,, and thus we 
may choose the FE approximation space somewhat conservatively; the latter would 
then accommodate the difference between the offline parameter train set Epp and the 
full (online) parameter domain P. In the remainder of this section we consider only 
I|U, (4) - Uy (H)Ily (and, briefly, |s, (4) — $y (m)l). 
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To begin, we consider the case in which efi = 0 and hence ŭy(4) = uy(H); for 
the purposes of this analysis, we explicitly remove the ~. We introduce the error e(u) = 
Up (H) — Uy(H) as well as the residual u + rp(; 4) € V' given by 


r (v; y) =f (vs) - a(uy(W), v3), Yv € Vj. (4.54) 
It is then standard to derive the error-residual relationship, 
a(e(H), v; H) = rap), Yv € Vy. (4.55) 
For future reference we also introduce the Riesz representation of the residual, 
Rp(B) = Rath H), (4.56) 


for Rp given by (4.13); we recall from (4.14) that II, (llv; = |R lly. 
We now define our a posteriori error estimator, y € P > Ay (H) € Ro,: 


IR llv 


; 4.57 
cK nite 


An(#) = 


where c;*?? qa) is a (nonnegative) approximation to c} (H). It is then a simple matter to 
demonstrate [33] that 


ch) AND Yn) 


< < : 4.58 
Sy) $ ely ~ ck? qa) Oe 


We observe from the left inequality that, if 0 < c/*P?(u) < chu), then lle(u)ly < Aya): 
Our error estimator is an error bound. We conclude from the right inequality that the 
error bound may overestimate the true error but by a factor which is bounded inde- 
pendent of N. 

We note that we did not in fact use any special properties of u,(u) in our deriva- 
tion of (4.58), and in particular we did not take advantage of the Galerkin projection. 
Hence, for the residual defined as (4.54) — with unperturbed f and a — our bound (4.58) 
in fact remains valid also for ŭy(4), and indeed for any function in V}. However, we 
shall see that for (efm + O and thus certainly) a nonaffine we cannot compute ||R,(W)|ly 
efficiently within the offline-online decomposition; more precisely, the operation 
count for evaluation of ||R;,(#)lly directly as (1,(H)X;,'1,(}))"? will not be indepen- 
dent of N,.? Furthermore, our “aggregate” error estimator (4.57) does not permit us 
to deduce, and hence control, the individual contributions of the RB approximation 
and EIM affine representation to the error |u (4) - ŭy(W@)lly. We next present an a 


3 We note, however, that at least for problems in two space dimensions (d = 2), this direct evalua- 
tion though not ideal is nevertheless feasible, in particular since the parameter-independent sparse 
optimally ordered X, can be Cholesky-factorized once. 
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posteriori error estimator for €fim + 0 which addresses these two issues. We reinstate 
the = notation. 

We first define ë(u) = up,(M) — ŭy (y); we next introduce the perturbed residual 
HH a(s H) € V' given by 


7 (Vs) = fv; p) — G( ity (p), v; u) (4.59) 


and the associated Riesz representation R (4) = Rpřņ(, 4). Our revised a posteriori 
error estimator is then given by 


lnl + fy (1 + Wty lly) 


! l (4.60) 
Ck (H) - efm 


Av@ = 


where ¢;*?P() is a (nonnegative) approximation to @,(). It can be shown that, if 
chim < ČP (p) < Qu, then |@@n)lly < Ay(u). We emphasize that the bound Ay(q) 
reflects — and thus can serve to efficiently control — both the RB and the EIM contri- 
butions to the error. Finally, we highlight the two constants which must be evaluated 
in (4.60): čp PP (u), to be discussed shortly, and ETs as introduced in (4.30). As re- 
gards the latter, we recall that we have direct control only over ep, and we must 
then assume that Epy is sufficiently rich to represent P. 


We also develop a simple a posteriori error estimator for our output: 
Isn(H) -Sn QI < (ËC Wy; + EEm)ÄN 4) + eflliay Gly: (4.61) 


Note for bea = 0, symmetric coercive problems, and compliant outputs — £(;u) = 
f(; p) — the bound (4.61) is demonstrably pessimistic; the latter is remedied by adjoint 
techniques [31], [33], which also provide for better approximation of noncompliant 
outputs. 


4.3.1.3.2 Approximate stability constant 

We recall that we wish to apply the error bound in the online stage. We shall show 
in the next section that the dual norm of the residual, which appears in the numer- 
ators of (4.57) and (4.60), in fact admits a very efficient offline-online procedure. It 
remains to develop a formulation for ¢,*P?(u) which also admits an efficient offline- 
online procedure and furthermore either rigorously, or plausibly, satisfies O < co < 
PP) < č), Vu € P. There are a variety of approaches [33]. In some cases, we 
can explicitly deduce č%®P? (u) in terms of the PDE coefficients: In Example 1.0, de- 
scribed by equations (4.9)—(4.11), for crz = O in our inner product (4.1), we may choose 
GPP (u) = min(1, 44) < a(u). However, for geometry variation, in particular in the vec- 
tor case, inspection no longer suffices; and for the noncoercive case the situation is 
even more difficult. Although there are approaches which can treat the general case 
rigorously, such as the successive constraint method [22], these techniques are unfor- 
tunately quite complicated and often prohibitively expensive in the offline stage. 
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The simplest approach might be to take eon (p) = (say) Cy (f)/2 [25]. If indeed 
Cy(H) converges to C)(u) as N increases, then this simple recipe provides, at least 
asymptotically, a lower bound. However, the RB spaces Vy are not designed to well 
approximate the stability constant [39]. We thus present the obvious extension: a co- 
lateral RB approximation for the eigenproblem associated with the stability constant. 
Since even an order-unity error in the stability constant will yield a good error esti- 
mator and indeed a bound, even a modest RB approximation should perform very 
well. This approach also serves several secondary purposes relevant to this handbook: 
a brief summary of RB treatment of (albeit somewhat nonstandard) eigenproblems 
[24]; further reinforcement of RB concepts. 

To begin, we recall the definition of the supremizer operators T,,(u) (respectively, 
Ty(u)): For any w € Vp, p € P & Ñ (0) € L(Vp Vp) such that, for any w € Vp 
(T,(W)w,Vv)y = (w,v;p), Yv € V, (respectively, for any w € Vy, p € P + Ty(p) € 
L(Vy,; Vy) such that, for any w € Vy, (Ty()w, v)y = a(w, v; 4), Yv € Vy). Here L(W, W) 
denotes the space of continuous mappings from W to W. We now introduce the fol- 
lowing generalized symmetric eigenproblems: For the coercive case, find y € P |> 
CF (H), An (H)) € Vp x R, such that 


5 (ACD. ¥5 4) + A(v,¥,(W)sH)) = AnDY) v)y W € Vis (4.62) 


and for the noncoercive case, find u € P + (®, (y), 0;,(H)) € Vp x R, such that 


(E DDR). TW) y = ROR, V) W € Vis (4.63) 


we enumerate the modes in order of increasing magnitude of the eigenvalue. Note 
that the inner product constant c,2 in (4.1) should be chosen large enough to ensure 
adequate separation of the lowest eigenvalues.* 

It is readily demonstrated that 


— AP) > Pr) y) 


On) = An): = Cg, Go), Fry oe 
and 
aoe ËR Tr WCnWM)dv 
Pr) = (on), | (CERD Or) ee 


We thus observe that a good approximation for the eigenfunction will yield a good ap- 
proximation for the respective eigenvalue. We can now proceed to RB approximation. 


4 We suggest a value c;2 = MaXyep max([[Yoo(-3 llr)» MaXie{1,2,3} Yat Wlre/P), where | is a 
characteristic minimum length scale associated with Q. 
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In particular, we can envision that, just as u, resides on a low-dimensional para- 
metric manifold, so do (¥;,), and (@,),. We may thus construct corresponding RB 
spaces and bases, respectively: For the coercive case, Vy and Vy for1 < N < Naai 
for the inf-sup stable case, Vy and Vy for1 < N < NẸ x The manifolds may not 
be smooth for eigenproblems; however, we can accommodate mode crossing through 
proper choice of a sufficiently rich RB space and in particular through incorporation 
of the first few eigenfunctions. We note that our interest here is in the eigenvalue, not 
the eigenfunction, and in particular the latter serves only to develop a good approxi- 
mation for the former. 

The Galerkin weak statements of the RB approximations directly follow: For the 
coercive case, find u € P > (¥y (H), Ày (H)) € Ve x R, such that 


5 (aC Ya), vs) + G(v, Wy(W)s M)) = AnH) (Yn (H) V)y Wwe Vy (4.66) 


and set ¢*PP(w) = (Ay(M)),; for the noncoercive case, we must consider y € P > 
(Dy (pH), Oo" (H)) € Ve x IR, such that 


(T DDNW, Tr(WV)y = On G)(On@)V)y We Vy, (4.67) 


and set ¢,*?? (qu) = (oy (p)); Note in (4.67) we retain T,,(u), and do not substitute Ty(p). 
In practice, we might even deflate (Ay(u)), and (oN (H)), by some factor, say, 1/2, to 
ensure that our estimates approach the true respective stability constants from below. 

We also provide here the associated discrete equations: (Wy(u) € R”, Ay (H) € R,) 
satisfies 


(VET 5(AnQo + ARQ) VY J YOD = An QOCVET Xn VA): (468) 


EN (H) 


similarly, (Oy(y) € RY, On (p) € R,) satisfies 


(Vi (An(W)X;An(H)) Vn) Pn = RVA | Xp Vy) Py (H)- (4.69) 
Ey) 


The RB matrices associated with these eigenproblems admit an affine decomposition: 
For (4.68), Ey H) can be expressed as 


Q 
S 1 

EÑ (H) = X oo ViTa + Ag vy), (4.70) 

q= 
which is the usual single-sum affine expansion; for (4.69), Ey (p) can be expressed as 

© ee i OT p GT y-1 ad \y7P 

END = > > 940004 (Vy (AF Xp AF Vy)» (4.71) 

q=1q'=1 


which is now a double-sum affine expansion. 
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4.3.1.4 Offline-online computational procedures 


We describe here the offline-online computational procedure. In the offline stage we 
prepare a parameter-independent data set. In the online (or deployed) stage we per- 
form RB queries: u +> (iy(H), Sy()). In general, the offline stage (respectively, a single 
online RB query) is very expensive (respectively, very inexpensive) relative to a single 
FE query u > ŭ, (y). As described in Section 4.1, the RB method, and in particular the 
offline expense, can be justified in the real-time context or the many-query context. 

In Section 4.4.1 we shall describe the procedure by which we identify our RB 
spaces. The latter is performed as part of the offline procedure. In the current section 
we presume that the RB spaces are given in the form of the RB basis matrix Vy . 
We consider here the offline-online procedure for subsequent (i) formation and so- 
lution of the RB discrete equations to obtain ŭy(4) and $,(), and (ii) evaluation of 
the dual norm of the residual, ||R,,(H) lly, as required by our a posteriori error indica- 
tor (4.60). More generally, the former is an example of a single-sum affine expansion, 
whereas the latter is an example of double-sum affine expansion. Other examples of 
single-sum and double-sum expansions include evaluation of (Ay(j)), and (oy(H))1, 
respectively, as required for the stability-constant approximation; the latter thus fol- 
low offline-online strategies very similar to those described below for the RB linear 
system. 


4.3.1.4.1 RB linear system: formation and solution 

In the offline stage, we form Ayo} <q< Qa Fro l < q < Qf, and Chal <q< 
Q, of (4.52); the operation count, taking into account FE sparsity, is O(Q,N?. Nn) + 
O(Q¢NmaxNn) + O(QeNmaxNp). It is important to note that we form these matrices and 
vectors for the largest space, VN vax? as the RB spaces are hierarchical, we can then 
readily obtain the matrices (respectively, vectors) for any other RB space, Vy, by sim- 
ply extracting the N x N first entries (respectively, N first entries). In the online stage, 
for any given N and u € P, we first form Ay(p), f w(H), and y(u) from (4.50) in 
O(Q,N’) + O(QrN) + O(Q,N) FLOPs; we then solve (4.47) for uy (4) in O(N?) FLOPs 
— in general, Ay(u) shall be a dense matrix; finally, we evaluate sy(u) from (4.48) in 
O(N) FLOPS. Note that if we wish to visualize the full field, then an additional O(NN;) 
FLOPs are required to evaluate the FE basis coefficients of the RB approximation as 
VyUy(H); the online operation count is independent of N, except for this (elective) 
full field reconstruction. Note however that, if the visualization of each RB function is 
stored and prepared offline as a frame on a GPU, only the linear combination of these 
frames is required and we are back to an O(N) complexity. 
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4.3.1.4.2 Residual dual norm evaluation 
To begin, we form the FE representation of 7,(-;}1), 7,(@’;),1 <j < Np: 


tn) = Fh) - Àn) V yuy (H). (4.72) 


It is then readily demonstrated from R (4) = RyFp_(sH) and (4.14) that the dual norm 
of the residual, hence ||R;,(W) lly, is given by 


IR Dl = Fr - A,(WV nun W) X; Fn) - A, (WV yuy (p)). (4.73) 


We now introduce Oy € R2*@" as 


On (H) = (Of)... OF! OG QD(Uy D) p -- -> OGOUN) y» 
02) (uy(H)) > «02 (W)(Uy))y) > (4.74) 


and also Ly € RM” +N) as 


Q 3 x 
Ly =( fh lee fy Anny l-1 -Åg Vy); (4.75) 
N,x1 N, xN 


note Ly is expressed in block column form with | as block delimiters. We now com- 
bine (4.73), the affine representations of the FE operators, (4.27), (4.74), and (4.75) to 
obtain 
5 3 1/2 
IRnQllv = (ONH) Ly X; Ly OND)“; (4.76) 
Wy 


note that Wy € R'&*2.%)*(Q+@.) We can now describe the offline-online decompo- 
sition. 

In the offline stage we form Wy, . In the (say) direct-solution context, we would 
perform a sparse Cholesky of (optimally ordered) X, once. We would then perform Qs + 
QaNmax forward/back substitutions to find Ly. = X; L Npa? WE would then complete 
the formation of WN ax by matrix multiplication, |e ae at cost O((Qr +Q,N )°Ny) 
FLOPs. Note that Wy is parameter-independent. In the online stage, given u € P and 
our associated RB approximation uy (4), we first extract submatrix Wy from Wy „„ and 
evaluate Oy(H). We then perform the sum (4.76), (OT (ul) Wy Oy (uy)! ? The operation 
count is O((Qr + QaN )) FLOPs, which we note is independent of Nae We observe that 


5 Wecan now readily define minimum-residual projection: Find üy (u) and hence ©* (u) which min- 
imizes lp) liy. It follows from our offline-online discussion that minimum-residual projection will 
be more expensive than Galerkin projection but only as regards formation of the RB linear system and 
in particular for larger Qf, Qa- 
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the operation count scales quadratically with both Q;, Q, and also N, which empha- 
sizes the important role of a posteriori error estimation to control both the RB but also 
the EIM costs. 

Finally, we note one shortcoming of the offline-online approach. Let us denote ma- 
chine precision by €p;ec. The construction (4.76) computes a small number, IRnQOllv. 
as the square root of the cancellation of (many) large summands. To illustrate the dif- 
ficulty, consider Eprec = 1x 101°, assume ||R;,(Wlly = 1x1077°, and furthermore say (for 
simplicity) that 0 (H) Wy On (H) is the sum of just two terms, respectively 1 + 5 x 10°70 
and -1+ 5 x 107%. Clearly upon truncation in finite precision we obtain for the sum 
(respectively, the square root of the sum) not 107% (respectively, 101°) but rather 0. 
This finite-precision effect can in principle compromise numerical convergence tests 
for sufficiently high accuracy, and remedies are proposed in the literature [11, 4]. How- 
ever, for engineering calculations, the limitation is not significant: It will very rarely, 
if ever, be the case that the data for an engineering problem are known to sufficient 
digits to warrant a numerical error as small as 1 x 10°°. 


4.3.2 Extensions 


4.3.2.1 Evolution problems: parabolic PDEs 


4.3.2.1.1 Galerkin projection 
As for elliptic PDEs, we are given a hierarchical set of RB spaces {Vy }y-1,...N,,,,- We are 
sponding basis matrix Vy € RY’, (Vy)in = (Che 1 sk < Ny, 1 <n < N.The 
method by which we shall develop Vy, „ for the parabolic case [20], related to but also 
different from the method with which we identify Vy, for the elliptic case, shall be 
summarized in Section 4.4.3.1. We note here only that the construction shall ensure 
that uọ (our initial condition) resides in Vy. 

We can now state the Galerkin projection [19, 20]: Given N € {1,..., Nmax} and 
H € P, we look for ity, (W)(= up(t;m)) € Vy, j =1,...,J, such that 


-j -j-1 

ni eee f vn) + (ity 4 (4), Vs H) = T(t)f(v;w), Yv € Vy; (4.77) 

we impose the initial condition iy) = Ug (€ Vy, by construction). We note that 

in (4.77) there is no reduction in the temporal dimension: the RB projection (4.77) re- 

tains the “true” finite difference discretization of the FE projection (4.39); the RB ac- 

celeration is effected solely through the dimension reduction in the spatial dimension. 
To develop the discrete equations we require the RB mass matrix, My(p) € me, 


(MND) = GSM), 1<kn<N, (4.78) 
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which we may form as 


Qn 
M(H) = } 04, (Mi, (4.79) 
q=1 


for Mi, = mig), ©, 1<k,n<N,1<q < Qm. We may directly formulate our RB mass 
matrix and constituents in terms of the corresponding FE quantities and our basis 
matrix: 


Myn) =VyM,(WVy, MI, =VẹyMIVy, 1<q< Qm (4.80) 


We can then state the discrete equations to be solved at each time t: 


(Ango + tv) ) ty = EMN aly) OF A80 
Éy (y) 


Note that, in general for RB methods, we prefer implicit temporal discretizations, 
since inversion of the small RB discrete operators is relatively inexpensive (and 
furthermore the RB mass matrix is not close to diagonal in particular given our 
V-orthonormalization of the basis) and implicit methods allow larger time steps. 


4.3.2.1.2 Error estimation 

We directly consider the more practically important case of a postenon error estima- 
tion. We shall provide here an estimator for the case in which cam = = 0 and hence we 
shall suppress the * for the purposes of this analysts extension to Efm # 0 is not diffi- 


cult. We first introduce the error e! (u) = Uh ap Uy, aD, 1 <j < J. We next define, for 


1<j</J, the residual pH r aCGH) eV', 


uy aH) - Uy ae J ) 
Vs 


At (wy a (D> v; u), Wwe Vip 


(4.82) 


r (VB) =1(’)F(vsp) m( 


and associated Riesz representation, R? i Pl) = Ra”? h, E ;H). We then define, for 1 < j < 


J, our error estimator Ay n (H): 


1/2 


Ht) = ( org Rha) (499) 


where c?*?P(w) is an approximation to ap(p), for example as developed in Sec- 
tion 4.3.1.3. It can then be shown [19] that, if cP? (4) < a(i), then 
1/2 


a Fi l 
(melon. can) «ea S vol <A j=.. (4.84) 


j= 
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We note that, consistent with the continuous problem formulation, we control both 
the (discrete-time) C (0, T]; L™Q)) error and the L7((0, T]; V)-error. 


4.3.2.1.3 Offline-online computational procedures 
As for the elliptic case, we presume here that the RB basis matrix, Vy, „„ is provided; 
we discuss the associated procedure, part of the offline stage, in Section 4.4.3.1. 

We first consider the formation and solution of the RB linear system. In the offline 
stage, we form Aj, „1< q< Qas Mx...’ 1<q< Qn: Fi, ,l<qs Qf, and ey jl< 
q < Q; of (4.52) ann (4.80); the operation count, taking ‘into account FE sparsity, is 
O((Q,+Qyn) NinaxNn)+O (Qf NimaxNy)+O(QeNmaxNp). In the online stage, for any given N 
and p € P, we first form y (y), My (9), Ñy (H), fy (4), and @y (pu) from (4.50) and (4.79) 
in O((Q, + Qm)N?) + O(Q;N ) + O(Q,N) FLOPs; we next perform the LU decomposition 
of Ñy (4), once, at cost O(N 3). we then solve (4.81) as J forward/back substitutions at 
cost O(N 2); finally, we evaluate sy(y) in O(N) FLOPs. Note that we take advantage 
here of the linear time-invariant nature of our operator: For J not too small, we expect 
the online cost of the J parabolic updates to scale as O(JN*) FLOPs, hence roughly 
independent of Q. and only quadratically with N. 

We next turn to the evaluation of the error bound, and in particular the contribu- 
tion of the dual norm of the residual. We now introduce o at € RU mNHAN 1<j<J, 
as 


Oka = (0100.-070. 


TA At ()) - (wy) At w) 1 TA At (H))N - (wy) At Hw 


1 
On) x oh) - a 
@ 2m p) TA u) - (wy) a) @2m(y ) (wy nD) - (wy) ac) 
m i At l MAN At 
Oa (t u D)p -> A 
T 
O RC E (4.85) 
and also Lyme € BRNnX(Q+(Qn+Qa)N) ag 
Lym = Fil. If - MV y |. |- M®Vy | -Alwy |... 1- A&Vy); (4.86) 


note Ly x is expressed in block column form with | as block delimiters. We now com- 
bine (4.73), the affine representations of the FE operators, (4.27), (4.85), and (4.86) to 
obtain 


12, 
) 


IR KDl = (Oha) 0D Lin Xp Lye Oy ne) (4.87) 


Wy,at 
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note that Wyp € R+ Ont QNH Ont), 

In the offline stage we form Wy ae- In the (say) direct-solution context, we would 
perform a sparse Cholesky of (optimally ordered) X; once. We would then perform Qs + 
(Qm +Qq)Nmax forward/back substitutions to find Ly Qe X; L Nra At? We Would then 
complete the formation of Wy „at by multiplication, Ly, ate, At? at cost O((Q- + 
(Qm + Qq)N)*N;,) FLOPS. Note that Wy,,,,.At 18 parameter-independent. In the online 
stage, given u € P and our associated RB approximation uy nD, 1 <j < J, we first 
extract submatrix Wya from Wy „at and evaluate Oh, ne G0,1 < j < J. We then perform 
the sum (4.87), COR Wynt Oh, 0D)” 2 1<j< J; the operation count (for given 
jjis O((Qr + (Qm + Qq)N )?) FLOPs, which we note again is independent of N}. 


4.3.2.2 Anonlinear elliptic problem 


4.3.2.2.1 Galerkin projection 

As for linear elliptic problems, we are given a hierarchical set of RB spaces {Vy c 
Vikn=t,..Nmax aNd associated basis G hit Nee We may then define our RB-Galerkin 
approximation for (4.41): Given N € {1,...,Nmax} and H € P, find ŭy (u) € Vy such that 


pde 


X po Vay (x02; u) 4 vo (xen) 
j=l 


+ D Bid mm Mn” w))] YE), voghee®) pmeee 


aad. 


=H $ ppt v(e), wee Vy, (4.88) 
j=1 


where the EIM interpolation system is defined in Section 4.2.3.3 (recall (4.44)). We may 
further introduce the Newton update equation associated with (4.88): For given cur- 
rent iterate ak (u) c€ Vy, find ôŭy (H) € Vy such that 


pwed 
pra Vôüy (xte y) . Tae) 
j=1 
M p% 
— ank d, ~k d, d, d, 
PD Oh a woe a E Ea apaa 
m,m'=1 K K j=1 


=*K(v3p), YveVy. (4.89) 


Note 7) is the derivative of 7 (we reserve prime for dummy indices): For example, for 
n(z) = 2, H(z) = 3z?, and for n(z) = |z|z, H(z) = 2\z|; in general, 7 > O from our 
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Spor of monotonic nondecreasing n. The right-hand side of (4.89), residual u € 
Pr 7K(- swe Vp i is defined as 


pede 
7K(v;m) = u X py (x R 
j=1 
aad. 
= 5 pe Vib (xa? EOE 
j=l 
4 k d,Q a d,Q d,Q 
= FD MOE) S E e we Vy. 
mm! =1 m j=1 


(4.90) 


The left-hand side of (4.89) is the Gâteaux derivative of our RB nonlinear operator. 
We now proceed to the discrete equations. We first introduce parameter-inde- 
pendent matrices and vectors By € RY", Cy € RY", and fi € RN given by 


pwed 
Byw = D pe wert (x (x y TAR ann aN, (4.91) 
j=1 
M pedo 

(Cy) min = >, (Bat) pam! (5), Gane) pase, 1cm <M,1<n<N, (4.92) 

m=1 j=1 

pede 

Ge De Ga (4.93) 


jal 


respectively. Note that By, Cy, and fy are all parameter-independent. 
Our Newton update may then be expressed as 


N 


S ONID) pn (CUN) w = HOF) Ly (By) nn’ (UN HD) 


n'=1 n'=1 


M 
- Ý nyw) (Cy)mn 1snsN, (4.94) 


m'=1 


where the Jacobian matrix JWD e RYN is given by 


M 
NH) mnt = (Bunn + > Ahm DCN mwn Se AS) 1ean <N. (495) 


l I 
m'=1 m 


Here NK mo HE mo 1<m' <M,are given by 


N 
thy wW = (X ÜK (H)), on) 1<m’' <M, (4.96) 
n=1 
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N 


ae (Sakit 


i 
n=1 m! 


ao 1<m' <M. (4.97) 


We note that for n(z) = z’? it can be shown that J% w) of (4.95) is symmetric positive- 
definite for en, sufficiently small. 


4.3.2.2.2 Error estimation 

We directly consider the more practically important case of a posteriori error estima- 
tion. We shall continue to assume (effectively) exact quadrature. By way of prelimi- 
naries, we define the Laplacian bilinear form b : V x V —> R, 


b(w,v) = | vw Vv, Yw,ve V’. (4.98) 
Q 


We then define (for consistency with earlier parts of the chapter) 


; (4.99) 


CO = iE viy 
in the current context, č, (4) is a coercivity constant which is in fact parameter- 
independent. 
We now define our error as é* (H) = up, (H) — Uy (H). We further introduce the Riesz 
representation of our residual, RK = Rp% (+ p) for 7K (+5 p) defined in (4.90). We may 
then define our a posteriori error estimator, 


IRRQDIly + 2! LOI"? toler 


xk EIM 
AK u) = x (4.100) 
cor (w) 
Here cz: is the weight of the L?(Q)-contribution to our norm || - ly, IQ] is the mea- 
sure of our domain in RÎ, and č% PP (u) is an approximation to the coercivity constant 


č (4), (4.99), as developed in Section 4.3.1.3. We can then show that, for any pt € ce 


and "PP (u) < GM), 


le qully < KW; (4.101) 


we can plausibly extend our result to any u € P, perhaps with a safety factor, depend- 
ing on the richness of ENL The a posteriori error bound (4.100), (4.101), identifies 
three contributions to the error: the (incomplete) convergence of the Newton iteration, 
as reflected in superscript k; the RB approximation error, as reflected in the dual norm 
of the residual; and the perturbation of our problem to affine form, as reflected in the 
FIM tolerance parameter. We emphasize that, within this nonlinear iterative context, 
the error bound can serve not only to assess and control RB and EIM errors but also as 
an iterative termination criterion. 
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We briefly derive this result in particular to emphasize the simplification afforded 
by our monotonic nondecreasing nonlinearity and conversely the challenges associ- 
ated with more difficult nonlinearities. We first note that (under our assumption of 
exact quadrature) our residual may be expressed as 


v; =H | v — b(ŭ% W, Vv) - EACE W € V}. (4.102) 
Qa à 


We also note that u, (4) satisfies 


b(u,(p), v) + [anwy =u | v, Wer). (4.103) 
a d 


It thus follows that 
b(un (H) - KD, v) + | [nun 0D) - N(R) 
Q 


= PRW; H) + | ZaclaCit <0) —n(ity(u))|v, Wve Vp. (4.104) 
Q 


We now choose v = up (u) - ak, (p) and note from our assumption of monotonic nonde- 
creasing 7 that 


finno) - nÜ WD )] [un - aK Gu)] > 0. (4.105) 


Q 


We then consider the interpolation error term and invoke the Cauchy-Schwarz in- 
equality and our definition of the norm | - |y; the result directly follows. We emphasize 
that without the property (4.105) we would need to estimate the inf-sup constant as- 
sociated with the Jacobian and then consider a contraction argument per the Brezzi- 
Rappaz-—Raviart theory [37] - hence much more difficult to realize in practice if we 
wish to quantitatively evaluate the necessary constants. 


4.3.2.2.3 Offline-online computational procedures 
As for the linear case, we presume here that the RB space and basis are provided. 

We first consider the formation and solution of the RB nonlinear system. In the 
offline stage, we perform the EIM for the nonlinear term and subsequently form By, 
Cy,,,,2 and fv... Then at each Newton iteration, we first form nk and ġ%, at cost O(MN) 
FLOPs. We next form the Jacobian at cost O(MN?) and the residual at cost O(MN). 
Finally, we invert the Jacobian at cost O(N 3) FLOPs. 

We next consider the a posteriori error estimator and in particular the dual norm 
of the residual. In fact, once we evaluate (nk m()) m=1,...m> the residual of our nonlinear 
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elliptic problem is computationally analogous to the residual of a linear elliptic prob- 
operation count for the dual norm of the residual is then O((N + M ot hence quite 
inexpensive; of course, if the weak form included other parameters, hence p > 1, the 
operation count would increase commensurately. 

We close this section by emphasizing the importance of hyperreduction. We first 
consider nonpolynomial nonlinearity, for example n(z) = |z|z: In the absence of hyper- 
reduction, the online cost would depend on N,, both for formation of the Jacobian and 
for evaluation of the dual norm of the residual. We next consider polynomial nonlin- 
earities, hence (z) = z for s an odd integer (in order to honor our monotonic nonde- 
creasing assumption): In the absence of hyperreduction, the online cost for formation 
of the residual is O(N**') and the online cost for evaluation of the dual norm of the 
residual is O(N’), and thus potentially prohibitive even for s = 3. We illustrate the 
difficulty for a polynomial nonlinearity n(z) = z’. In this case the nonlinear contribu- 
tion to the residual is given by 


N N N f r: m 
YY Y yo) GD) yen OD) [GRRR 6100) 
Q 


n=1 n'=1 n" =1 
form offline 
evaluate online 


where g" plays the role of test function. 


4.4 Approximation spaces 


4.4.1 Elliptic problems: weak greedy method 


Perhaps the most simple — and the most popular — approach to the identification of 
reduced-order approximation spaces is the proper orthogonal decomposition (POD).° 
It consists in building a matrix with entries (u(y), ün (u’))y, p, and p’ belonging to 
some subset pop of P with cardinal Kpop (large enough since Epop is supposed to 
scan well P), and considering the eigenvectors associated to the largest values of its 
largest eigenvalues. The interest of this approach is two-fold: 
— itis very simple to implement and does not rely on further mathematical ingredi- 
ents as is required in other methods (such as error estimators for the weak greedy 
approach that will be explained next); 


6 The POD can also be interpreted in terms of singular value decomposition, principal component 
analysis, or even Karhunen-Loève transform. 
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— it provides some hope that an RB exists (when the eigenvalues’ decay is fast 
enough), but also, on the contrary, a proof that the Kolmogorov N-width is (in 
some bad situations) large since dy(M, V) is always larger than )’,...y OF. 


However, in the parametric RB context, the POD has several drawbacks: 

— formation of the covariance matrix requires many (Kpop) queries to the FE approx- 
imation, u + t,(y); identification of the POD modes requires solution of a dense 
Kpop X Kpop eigenproblem; 

- the norm in which the approximation is optimal is L?(P; V), which is not ideal for 
pointwise (in parameter) queries, 4 > ŭy(Ņ). 


Alternatives, widely used in the RB context, are the greedy and (because more 
amenable for implementation) the weak greedy approach: The optimality norm 
L® (P; V) is more appropriate, consistent with the Kolmogorov N-width definition; 
only Nmax FE queries are required to identify the RB spaces {Vy}y-1,_w,,,,3 the result- 
ing approximation space nevertheless reflects information from Kya) > Nmax Points 
on the parametric manifold. The POD is discussed in depth in several other chapters 
in this handbook; we focus here on the greedy-type methods. 

We first present the algorithm: Let Eia] be a given subset of P with cardinal Kyria 
(large enough since 4,4) is supposed to scan well P). 


Algorithm 4.2: Greedy method: We assume that the set {U;(H)}ucz,,,, is not em- 
bedded in a small finite-dimensional space. 
Data: Ey {41 Ún : P > Vip ll- IIGreeay> tolgreedy 
Result: M, {Hm € Etiath<mem {6° € Valicicms Vm © V, dimVy = M 
1 Set M = 0, Vo = Ø, and err = o9; 
2 while err > tolgreeay dO 
3 Set M — M +1; 
4 Find My = arg sup yes... dn) - Hm-14hW)llGreeay (where My; denotes a 
projection approach — like an orthogonal projection in V or a Galerkin 
projection — onto Vy_; according to || - IIGreeay ); 


5 Define ¢! = (i, (yy) - Tmt Hy) [Ili Hu) - Tin) Ncreeay’ 

6 Update Vņų = Span{¢",i=1,...,M}; 

7 Set err = |[Up_(My) — Im-1ŭHUm)llGreedy’ 

8 end 

The pure greedy method is associated to the choice Egia] = P and ||- IlGreeay = ll- lly, the 


projection operator being the V-orthogonal projection. It is a theoretical algorithm as 
the computation of u* on line 4 above is quite impossible to determine as it requires in 
particular the knowledge of every i, (u), for any 4 € Epia Alternatively a weak greedy 
approach can be implemented where the exact V-norm is replaced by a surrogate, 
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an example being given by the a posteriori error estimate for Galerkin-RB approxima- 
tions, as the ones that have been presented in Section 3 (see [38] for the first actual 
use of this weak greedy approach). From (4.58), we can thus interpret the weak ver- 
sion with respect to the pure greedy method by replacing line 4 above by 


Select a, such that |ú (Hm) - Hm-1ŭrHUm)lly = ysup lür) - Hm- W ly (4.107) 
pe 


with some given y, such that O < y < 1 (in this context, the denomination “weak 
greedy” was first introduced in [8]). Another element that enters in the weak greedy 
is the choice Egia C+ P that should be large enough and well chosen so that (4.107) 
remains true. 


4.4.2 Optimality of the weak greedy method 


In this subsection we state recent results about the performance of the weak greedy al- 
gorithm compared to the Kolmogorov N-width optimal choice given by dy(M, V). First 
of all, we define oy(M, V): a comparable quantity with respect to dy(M, V) which is 
associated to the series of spaces {V,,},+9 defined in the greedy algorithm 


0,(M, V) = dist(™M, V,,) = sup inf l u) - valiy (4.108) 
H VYnEVn 


that characterizes the approximation space resulting from the weak greedy algorithm. 

Of course, if (0,(M,V))ys9 decays at a rate comparable to (d (M, V))n>o; this 
means that the greedy selection provides essentially the best possible accuracy at- 
tainable by n-dimensional subspaces. The first comparison between (0,(M, V))ns0 
and (d (M, V))ns9 was given in [9]: 0,(M,V) < Cn2"d,(M, V) which, of course, is 
only useful if d,(M,V) decays to zero faster than n™2™. A more conservative esti- 
mate results from the series of papers from [8] and [14]. We present here their proof in 
the Hilbertian context. 

For any choice Eyia] with cardinal Kyiaı > O large enough, let us consider the set 
Xk ia Of all vectors Up (4), H € Eyiai- It is included in M and thus we derive obviously 


Vm >0, dmXkup V) S dM, V). (4.109) 


This means that there exist m vectors b4, b», ..., bm, generating a finite-dimensional 
space Hm such that 


max min | H) -Cylly < dm(M, V). (4.110) 


HE trial Cm EHm 


By taking the projection of these vectors b4, bz, .. . , bm in Span(X;, ,,), we can determine 
m orthonormal vectors in Span(X;, ,,), which we denote by b,, b>, ..., Dn that generate 
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a finite-dimensional space H, c Span(X Kia) Of dimension m’ such that 


max min |ú 4) -—Cylly < dm(M, V). (4.111) 


HE“trial CmEHm 


Let us now introduce the scalar products a,; = (Üu) © ys 1< jn < Kyra 
where ¢ are defined on line 5 of Algorithm 4.2). We easily get a,,; = O for j > n and 
Up;(U,) = 0 y,j6;- The matrix A with entries a,; is lower triangular and incorpo- 
rates all the information about the weak greedy algorithm on M. In addition, what- 
ever the definition of the space V, this matrix leads us to an analysis in ¢7(N) (even 
€°({1, 2,..., Kiiai}) ) more simple than the analysis in V. From the very definition of the 
greedy selection, we get the two following properties: 

P,: The diagonal elements of A satisfy yo (M, V) < |annl < On(M, V). 
P,: For every m > n, one has Djin amj < 0,(M, V}. 


For any K and m, 1 < m < K, and any M = O, let us consider the K x K matrix G = 
(gij) extracted from A by considering the rows and columns of A with indices from 
{M + 1,...,M + K}. Each row y; of G is the coordinate of the projection of t,(My,,;) 
in the vector space spanned by m+1 S42--->m+x- Similarly as what we have done 
on the projection of t,(My,,;) in the vector space spanned by (741, m+2-- -> Six» WE 
project each vector b,,b,...,b,, in the vector space spanned by m41 Qm42-- -> Oak 
and further normalize them. The associated vectors are denoted as b;, 1 < i < mand 
span a finite-dimensional space Hm that satisfies, from (4.111), 


max min |ly;— Cmlle < dm(M, V). (4.112) 


l CmEHm 
Now we note that the £°-norm of each y; is, from property P, above, 
lyile < Oy(M, V). 


The projection of each of these vectors on the vectorial space spanned by b;, 1l<i<m, 
belongs to a ball (in dimension m) of radius < oy(M, V): B(O, oy (M, V)), due to the 
m-width property (4.112), each of the vectors y; thus belong to a “cylinder” with basis 
B(0, 0y(M, V)) and height < d,,(M, V) (in dimension K—m). The volume of the convex 
set spanned by these vectors y; (that is equal to the determinant of G) is thus upper 
bounded by Vn = Vol(B(0, o4(M, V))) x dn(M, yy. Recalling that the matrix G is 
lower diagonal, we thus get from P; 


det G = tM sldnnl < Vm < COy(M, V))™ x dp (M, V)", (4.113) 


7 Note that if the projection of Hm in Span(Xx,,,.,) is of dimension < m, we choose any vector bin 
Span(Xx,,,,,) to complement the family {Dihiciem: 
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where C is here a universal constant bounding the volume of the unit-ball, whatever 
the dimension (we can take for instance C = 6). 
We thus get (a slightly improved version of) Theorem 3.2 in [14]. 


Theorem. For the weak greedy algorithm with constant y in a Hilbert space V and for 
any compact set M, we have the following inequalities between o, = 0,(M,V) and 
d, = d,(M,V) foranyM>0,1<m<kK: 


MK omi < oy “oud. (4.114) 


Then, as of consequence of various choices of the values M, K, and m, we get the 
following corollary. 


Corollary. For the weak greedy algorithm with constant y in V, we have the following: 
— Foranyn 2 1, we have 


1 m 
On(M,V) < 6*y* min dy’ (M, V). 


In particular 0>,(M, V) < V2y"' Vd,(M, V), n > 1. 
- If d,(M,V) < Con“, thena,(M, V) < Cin“ with C, = 2**y Co. 
- If d,(M,V) < Coe" , then o,(M,V) < y2Coy te ™®" , where c, = 21 **cp. 


For the first item, we take M = 0, K = n, and any 1 < m < nin the previous 
theorem, and use the monotonicity of ø and the fact that op < 1. For the proof of the 
two other items we refer to [14]. 

As a final obvious remark we want to point out that dm( M, V) depends on M and 
thus on P. This suggests a localization greedy procedure so as to lower the effect of the 
large size of P; we refer to [16, 2, 29] for various approaches to exploit this argument. 


4.4.3 Extensions 


4.4.3.1 Evolution problems: parabolic PDEs 


The greedy method for elliptic problems has been extended to the parabolic case in the 
POD greedy method first proposed in [20]. On line 4 of Algorithm 4.2 we still identify 
Hm as the “worst-approximated” parameter value, but now the error bound is typically 
evaluated at the final time, T. The update on line 5 is then given as some number of 
POD modes associated with the projection error. 


4.4.3.2 A nonlinear elliptic problem 


For our particular nonlinear problem and associated error estimator the weak greedy 
Algorithm 4.2 requires little modification. More generally, apart from difficulties asso- 
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ciated with error estimation, the weak greedy does extend relatively easily to the non- 
linear case. However, as already noted, the EIM preparation of the nonlinear problem 
can be expensive, and for this reason more advanced techniques — which combine the 
EIM and RB greedy algorithms — are an active area of research (e. g., [13, 6]). 
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Alfio Quarteroni 


5 Computational bottlenecks for PROMs: 
precomputation and hyperreduction 


Abstract: For many applications, the projected quantities arising from projection- 
based model order reduction (PMOR) must be repeatedly recomputed due to, for 
example, nonlinearities or parameter dependence. Such repetitive computations con- 
stitute a performance bottleneck. Specifically, they limit the ability of a projection- 
based reduced-order model (PROM) to deliver the coveted speedup factor. This chap- 
ter reviews several state-of-the-art approaches for mitigating this issue and organizes 
them into two categories. Methods in the first category divide the computation of 
projections, whenever possible, into two parts: one that is responsible for the afore- 
mentioned bottleneck but can be precomputed offline; and another part that must be 
repeatedly performed online but whose computational complexity scales only with 
the dimension of the PROM. Methods in the second category are known as hyperreduc- 
tion methods: They achieve the desired computational efficiency by approximating 
either the quantity to be projected, or the result of the projection. The significance of 
hyperreduction for PMOR is highlighted using four numerical applications that are 
representative of academic and real-world problems. 


Keywords: empirical interpolation, energy conserving sampling and weighting, hy- 
perreduction, nonlinear model order reduction, reduced mesh 
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5.1 Introduction 


There are two main purposes for performing model order reduction: 
— Obtaining a low-dimensional model with minimum storage requirements in order 
to enable online computing in general and embedded computing in particular — 
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that is, computing inside a complete device that may include hardware and me- 
chanical parts. 

Obtaining a low-dimensional model with minimum processing time requirements 
in order to enable numerical predictions in real-time, or at least near real-time. For 
time-independent predictions, real-time is defined here as the shortest possible 
wall-clock time. For numerical predictions in the time interval [t?, T], where t? > 0 
denotes an initial time and T > t? denotes a final time, real-time is defined here as 
in T — t? units of time: For example, simulating 1 second of flow around an aircraft 
in 1 second wall-clock time. Furthermore, this definition of real-time is expanded 
throughout the remainder of this chapter to include near real-time, in order to 
avoid repeating after each occurrence of the words “in real-time” the alternative 
words “or near real-time.” 


While both objectives stated above are typically desirable, the second objective im- 
plies in practice the first one. Unfortunately, the reciprocal is not true, as explained 
below. 


For this purpose, the focus is set in this chapter on the parametric, time-depen- 


dent, semi-discrete, N,-dimensional, full-order computational model (FOM) 


l My, (w)Uy, (ts Hp) + fy, (Un, (t; y); p) = SN, (t; y), (5.1) 


Uy, (0; H) = Uy, (W), 


where: 


t denotes time, and the dot denotes a time derivative. 

H € P c R” denotes a parameter vector of dimension p, and P denotes the bounded 
parameter space of interest. It can be physical in the sense that it can pertain, for 
example, to initial conditions, boundary conditions, problem geometry, and/or 
material properties. It can also be nonphysical, for example, if u is a realiza- 
tion parameter or a hyperparameter vector associated with a stochastic process 
[36, 23]. 

My, (H) € RM is the parametric, mass matrix of this parametric FOM. For sim- 
plicity, but without any loss of generality, My, is assumed here to be a symmetric 
positive definite (SPD) matrix. 

Uy, (6 H) € R^ is the parametric, time-dependent, semi-discrete solution vector 
modeling the state of the system represented by this FOM. It is also often referred 
to as the vector of degrees of freedom (DOFs). 

uy, (u) € R®™ is an initial condition for Uy, (6 p). 

fy, (Uy, (t; y); H) € R™ is a parametric, nonlinear function referred to here and 
throughout the remainder of this chapter as the nonlinear, internal force vector, 
and its Jacobian 


ofn, 
Ouy, 


h 


Ky, (Uy, (t; p); p) = (Uy, (t W; p) € R0 ™ (5.2) 


5 Computational bottlenecks for PROMs: precomputation and hyperreduction —— 183 


represents the parametric, tangent stiffness of this FOM. In the linear case, 


fy, (Uy, (6 H); 4) = Ky, Huy (6s p), (5.3) 


where Ky, (H) is a parametric but otherwise constant matrix representing simply 
the stiffness of the parametric FOM (5.1). 

- Sm, (t; u) € R^» is a parametric, time-dependent vector referred to here and 
throughout the remainder of this chapter as the external force vector. 


Let 


uy, (6B) = Vu,(t y) Vte (t°, T], Ve RNP 


, u, € R”, n & N}. (5.4) 
In the above subspace approximation, V is referred to as the right reduced-order basis 
(ROB): It is typically orthonormalized with respect to some inner product for condi- 
tioning reasons. The vector u, is referred to as the vector of reduced or generalized 
coordinates. 

Let W € IR%**" denote the left ROB associated with V and orthogonal to it — that 
is, 

T 
WV = Ip (5.5) 

where the superscript T designates the transpose of the quantity to which it is applied, 
and Iy, € IRs denotes the identity matrix of size N}. 


The projection-based, Petrov-Galerkin (W, Y), reduced-order model (PROM) as- 
sociated with the FOM (5.1) and the subspace approximation (5.4) can be written as 


| M, (Mu, (6 H) + fn (U, (t; H); H) = 8nl6 p), (5.6) 
u,,(0; 4) = un), 
where: 
— The reduced matrix 
Mn (H) = W" My, ()V € R” (5.7) 


is the parametric, mass matrix of the above parametric PROM of dimension n « 
Nj); therefore, it is the reduced mass matrix. 
— The reduced vector 


f (u(t; W; H) = W' fy, (Vu, (t; W; p) € R” (5.8) 


is the parametric, reduced, nonlinear, internal force vector. From the definition of 
the tangent stiffness given in (5.2) and that of the subspace approximation (5.4), 
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it follows that the reduced Jacobian matrix K,,(u,(¢; 4); p) = % (u(t; W); p) is 


related to the full-order Jacobian matrix Ky,(Vu,(t; 1; H) by 
Of, _ wE ofn, sw ofy, OUN, 


K = 
"Ou, ou, ðUy, OU, 


= W'Ky,VeR™". (5.9) 


Hence, the reduced Jacobian matrix K,,(u,,(t; 4); p) represents the parametric, re- 
duced tangent stiffness of the parametric PROM (5.6). In the linear case where (5.3) 
holds, K,,(#) is parametric but otherwise constant — because in this case Ky, in 
(5.9) is parametric but otherwise constant — and represents simply the parametric 
stiffness of this PROM. 

— The reduced vector 


g(t; H) = W'gy, (ts p) € R” (5.10) 


is the parametric, time-dependent, reduced, external force vector. 

- wu? (u) = W'uy, (u) € R” is the initial condition for u,,(t; 4) associated with the 
initial condition for uy, (t; 4): Its expression results from the subspace approxi- 
mation (5.4) and the orthogonality condition (5.5). 


The counterpart of the PROM (5.6) based on a Galerkin projection is obtained by setting 
We=V. 


5.1.1 Computational bottlenecks 


First, consider specifically the linear instance of the FOM (5.1) — that is, the case where 
fy, (Uy, (6 u); H) = Ky, (#)Uy, (6 p) - which can be written as 

l My, (Uy, (t; y) + Ky, (Wun, (t; y) = Sy, (ts A) (5.11) 

Uy, (05 H) = Uy, D). 


From (5.3) and (5.7)—(5.10), it follows that the PROM (5.6) can be written in this 
case as 


(W" My, WV) ùt; H) + (W" Ky, (WV) Un(ts p) = W'8y, (6 W, 
M, (u) K, (p) Sry) (5.12) 
u,,(0; u) = W" uy, H). 


The real-time performance of the processing of the above parametric, linear PROM 
hinges on the real-time evaluation of the reduced matrices M„(4) and K,„(4), and re- 
duced vectors g,(t; p) and u,,(0; u). However, the computational complexity of the 
evaluation of each of these reduced matrices and reduced vectors, which must be re- 
peated every time p is varied, scales as © (Npn) and O (Nn), respectively. In other 
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words, this computational complexity scales with integer powers of the size N, of the 
FOM. For large values of N,, this prohibits the evaluation in real-time of the reduced 
quantities, even when the dimension n of the linear PROM (5.12) is very small. 

Nonlinearity aggravates the computational complexity issue raised above for 
parametric, linear PROMs in the following sense. Even in the nonparametric case, 
achieving a low dimension n « N, for a nonlinear PROM such as (5.6), while meeting 
accuracy requirements, does not suffice to enable real-time numerical predictions. 
To highlight this aggravation, which of course persists in the case of the parametric, 
nonlinear PROM (5.6), consider next the nonparametric instance of the nonlinear FOM 
(5.1). Because in this case M is independent of u, it is more convenient to construct 
the left ROB W such that it satisfies 


W’ My, V = In (5.13) 


instead of the orthogonality condition stated in (5.5). Indeed, from (5.7), (5.13), and 
(5.8), it follows that the expression of the nonlinear PROM (5.6) simplifies in this case 
to 


Intnl) + W’ fy, (Vu,(t)) = Wy, 
f,(u,(0)) 8, (t) (5.14) 
u, (0) = W' uy, . 


There are two main approaches for processing the above nonparametric, nonlinear 

PROM - namely, the explicit approach and the implicit approach: 

- Processing the PROM (5.14) using an explicit time integration scheme requires 
the evaluation at each time step of the reduced, nonlinear, force balance vector 
W' (fy, (Vu, (t))- SN, (t)). Again, the computational complexity of this evaluation, 
which must be performed at least once at each time step of the time integration 
process, scales as O (N,n). Therefore, it scales with the size of the FOM. Even when 
the dimension n is very small, this prohibits the processing of the PROM (5.14) in 
real-time (for example, see [21] for wall-clock timings associated with realistic en- 
gineering computations). 

- On the other hand, processing the PROM (5.14) by an implicit time integration 
scheme gives rise at each time step to a system of nonlinear algebraic equations. 
Solving this system of equations by the Newton method or a variant requires re- 
constructing at each Newton iteration, within each time step, the reduced tan- 
gent stiffness matrix K,,(u,(t)) = W' Ky, (Vu,,(t))V (see (5.9) ) in addition to the 
reduced, nonlinear, internal force vector. The computational complexity of each 
reconstruction of K,,(u,,(t)) scales as O (N, n n). Again, this prohibits in general the 
processing of the nonparametric, nonlinear PROM (5.14) in real-time. 


The computational issue highlighted above also hinders the real-time performance of 
time-independent PROMs in almost the same way. Indeed, for steady-state (or static) 


186 —— C. Farhat etal. 


problems, the parametric, linear PROM (5.12) simplifies to 


(W" Ky, (OV) upu) = W' gy, (H) (5.15) 
K,() Sn (H) 


and the nonlinear PROM (5.6) simplifies to 


W' fy, (Vu 0; H) = W' gy, 0). (5.16) 
fa (u, (4); p) Sn (HB) 


In the case of the parametric, linear, static PROM (5.15), the solution for each different 
parameter vector u* sampled in P of the corresponding reduced problem (5.15), in or- 
der to determine u,,(u"), gives rise to the same computational bottleneck associated 
with rebuilding the reduced matrix W' Ky, (u*)V in © (N,’n) operations. In the case 
of the counterpart nonlinear PROM (5.16), the same computational bottleneck arises 
at each iteration of Newton’s method — or a variant of this method — applied to the so- 
lution of the underlying nonlinear problem, during the update of the tangent stiffness 
matrix K,,(u,,), whether the nonlinear PROM (5.16) is parametric or not. 


5.1.2 Solution approaches 


Two major approaches have been developed so far for addressing the computational 
bottlenecks associated with the repeated reconstructions of a PROM highlighted 
above. Both of them share the offline-online paradigm that underlies most if not 
all PROM computations. Both approaches can also be described, broadly speaking, 
as divide-and-conquer approaches. 

The first approach divides the computation of the reduced quantities, whenever 
possible, into two parts: one part that is responsible for the aforementioned compu- 
tational bottlenecks and can be precomputed offline; and another part whose com- 
putational complexity scales with integer powers of the small size n of the PROM and 
therefore can be processed online and in real-time. This approach is both feasible and 
exact for at least two different classes of problems: the class of parametric, linear FOMs 
(5.11) admitting an efficient parameter-affine representation; and the class of nonlin- 
ear FOMs with a low-order polynomial dependence of the internal force vector on the 
solution (see (5.22)) and a time-independent external force vector. 

By contrast, the second major approach for achieving the same objective can be 
characterized as an inexact approach, where an additional layer of approximations is 
introduced into the PROM construction process in order to enable the evaluation of all 
reduced quantities in real-time. This approach includes the computational paradigm 
known today as hyperreduction. 
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5.1.3 Chapter organization 


Throughout this chapter, the parametric FOM of interest is assumed to be reducible. 
Therefore, the objective is set to squeezing the most performance out of the corre- 
sponding PROM. To this end, this chapter focuses on presenting the state of the art 
of both major approaches outlined above for mitigating if not eliminating the compu- 
tational bottlenecks highlighted in Section 5.1.1. It is organized as follows. 

Section 5.2 reviews the notions of global and local ROBs, and their associated 
global and local PROMs. Section 5.3 presents the first approach for addressing the 
computational bottlenecks associated with the repeated reconstructions of a PROM 
highlighted above. Section 5.4 reviews the second approach, and more specifically, 
two methodologies that belong to it. The first one targets parametric, linear PROMs. It 
is based on the concept of a database of pointwise linear PROMs, and that of interpo- 
lation on matrix manifolds. The second methodology is known as hyperreduction. It 
is equally applicable to parametric and nonparametric, linear and nonlinear PROMs. 
Section 5.5 illustrates two of the most successful hyperreduction methods with both 
academic and real-world, parametric and nonparametric, linear and nonlinear appli- 
cations. Section 5.6 summarizes and concludes this chapter. 


5.2 Global and pointwise ROBs for parametric 
PROMs 


Because of the sheer number of computations (or simulations) they imply, parametric 
numerical predictions are a major source of motivations for projection-based model 
order reduction (PMOR). However, they simultaneously constitute a source of compli- 
cations for the construction of a PROM that is robust with respect to variations in the 
parameter vector u. A popular approach for addressing this issue is to construct a sin- 
gle right ROB V for which the approximation (5.4) is accurate in the entire parameter 
space of interest P — that is, 


Uy, (6; H) = Vu, (ts) vee [t°, T], Yue Ps Ve Ru, €R", n«Nyp (5.17) 


In this case, the right ROB V is typically called a global right ROB. It is constructed by: 
sampling many points in P using any sampling procedure, but preferably a greedy 
procedure; generating one or multiple solution snapshots at each of these points; and 
compressing them if needed or desired to construct Y. Note that for a Petrov-Galerkin- 
PROM, the construction of a global right ROB V calls for the additional construction 
of a corresponding global left ROB W. 

While it is particularly effective in the nonlinear setting, the global ROB approach 
is equally valid in the linear one. In this sense, it is a comprehensive approach. How- 
ever, if P is high-dimensional and/or the solution of the parametric FOM is highly 
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sensitive with respect to variations in u — in which case the solution manifold of the 
governing FOM equations has a large Kolmogorov n-width — the global ROB approach 
may be inefficient or simply unfeasible. Indeed, the dimension n of V (and, if appli- 
cable, W) required so that the subspace approximation (5.17) delivers the desired ac- 
curacy may be too large in this case to enable a PROM to achieve its “compactness” 
(minimum storage requirement) and performance objectives. In this case, an alter- 
native approach for addressing the parameter dependence of a PROM based on the 
concept of pointwise ROBs may be considered. In this other approach, a set of param- 
eter vectors y; is properly sampled in the parameter space P — for example, using also 
a greedy procedure - and a pointwise right ROB Y; = V(y,) (and if applicable, a left 
ROB W; = W(y,)) is constructed at each sampled parameter point u; € P. Section 5.4.1 
presents an efficient computational strategy for exploiting this concept of pointwise 
ROBs in the linear setting. 

Because the concept of a global ROB is feasible in both linear and nonlinear set- 
tings, but that of pointwise ROBs — which so far has been supported by interpola- 
tion at queried but unsampled parameter points — has been exploited to date only in 
the linear setting, the following assumption is made throughout the remainder of this 
chapter: Whenever the context is set to that of a parametric PROM, a right ROB Y (and 
if applicable, the corresponding left ROB W) is assumed to be a global ROB, unless 
otherwise specified (as in Section 5.4.1). 


5.3 Exact precomputation-based methodologies 


The solution approach for eliminating the computational bottlenecks associated with 
the repeated reconstructions of a PROM characterized in Section 5.1.2 as an exact ap- 
proach is overviewed here in two different representative contexts: that defined by a 
special class of parametric, linear FOMs; and that defined by a special class of para- 
metric or nonparametric but nonlinear FOMs. In both contexts, this approach consists 
in reorganizing the exact computation of the reduced matrices, tangent matrices, and 
vectors, as applicable, in two parts: a first part that is responsible for the aforemen- 
tioned bottlenecks and can be precomputed offline; and a second part whose com- 
putational complexity is independent of N}, and whose real-time evaluation is feasi- 
ble. 


5.3.1 Linear FOMs and efficient parameter-affine representation 


Consider again the linear instance (5.11) of the parametric FOM (5.1). Note that what- 
ever is the dependence of My,, Ky, and gy, on the parameter vector u € P, these two 
matrices and vector can always be expressed as follows: 
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im 
My, (H) = Moy, + > m(W)Mi, » 
i=l 


Ky, H) = Ko, + Py kK,» (5.18) 
i=1 


By, (6 W = Bo, (0+) 8:008, (©. 


i=1 


where M, € RY), i =0,...,iy3 Kp € RWM, i=0,...iki 8gp €R, i= 0... ig 


each of the scalar functions m;(H), kW, and g;(u) describes the dependence of M; > 
h 
K; and g; on p, respectively; and im < Np’, ix < Nj’, and ig < Np. 
h 
In the context of the two-sided Petrov—Galerkin projection of the parametric, lin- 
ear FOM (5.11), the parameter-affine representation (5.18) of the otherwise arbitrary 
dependence of this linear FOM on pis parameter-preserving, in the sense that it eases 


the reduction of the FOM (5.11) to a PROM that has the following similar form: 


iy 


M, (wu, (ts p) + K,, (wu, (t; BP) = Zn(t; H), 


u, (0; 4) = u(y), 
where 


im 
T T 
M, (H) = W' Mo, V +9 mi) WM, V> 


precomputable a precomputable 
eR?" eR” 
č (5.19) 
T T i 
KD = WK, V +Ù KOD WIKY , 
eee e K 
precomputable = precomputable 
€ Ron E Ren 


Suits H) = W'8o,,(0 +) si) W's), (0 » 


=1 ue 
precomputable i precomputable 
eR" eR" 


un (H) = W' uy, H). 


From (5.19), it follows that all building blocks of the reduced quantities M,, K,,, 
and g,,(t) — that is, {WI MV}, (WIK; V} and (WSO — can be precomputed 
offline; for any queried parameter point uw“ € P, the reduced matrices M,,(u") and 
K,,(u*) can be computed in © (i,yn”) and O (ign?) operations, respectively; and for 
any queried parameter point p* € P, the reduced vector g,,(t; p“) can be computed in 
O (ign) operations. 

Hence, the reduced quantities M,,(u*), K,,(u"), and g,,(t; w*) — and therefore the 
parametric, linear PROM (M,,(u"), K,(u"), g(t; u*)) - can be computed in real-time 
only if 

im < Ni, ix «Nj, and i 


g <M, (5.20) 
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in which case the linear FOM (5.11) is said to admit an efficient parameter-affine rep- 
resentation. In other words, the representation (5.18) and the conditions (5.20) define 
the class of parametric, linear FOMs whose associated Petrov—Galerkin (W, U) PROMs 
can be processed online and in real-time using the exact method for treating the pa- 
rameter dependency highlighted in (5.19). 


Remark 5.1. Given a linear problem with a complex parameter dependency, the con- 
ditions (5.20) may or may not hold, depending on the level of fidelity of the modeling 
of this dependency. For example, consider the case of the finite element (FE) modeling 
of a geometrically complex and highly heterogeneous mechanical structure. If the ge- 
ometric and material properties of such a structure are homogenized, or at least piece- 
wise homogenized, the conditions (5.20) will hold. On the other hand, if these prop- 
erties are represented in the computational model with the highest possible level of 
fidelity and such a representation leads to an FE model where each element has differ- 
ent values of the geometric and/or material properties, conditions (5.20) will not hold. 
In this case, the modeling of the parameter dependency could be simplified to the ex- 
tent where the additional modeling errors induced by such a simplification are of the 
same or a lower order than any other modeling error — and particularly, the PMOR error 
— and the conditions (5.20) will be satisfied, in order to enable real-time processing. 


5.3.2 Nonlinear FOMs with polynomial dependence on the 
generalized coordinates 


Consider next the parametric, nonlinear FOM (5.1) and its associated PROM (5.6). As 
discussed in Section 5.1.1, the real-time processing of this parametric, nonlinear PROM 
faces two computational bottlenecks: the reevaluation at each queried point uw € P 
of the reduced matrix M,,(u*) = W'My, (u*)V, which requires © (N; n) operations; 
and the evaluation at each explicit time step of the reduced, nonlinear, internal force 
vector (5.8), which necessitates O (Nn) operations, or at each Newton iteration of each 
implicit time step of the associated Jacobian (5.9), which requires © (N,’n) operations. 
The first bottleneck was addressed in Section 5.3.1, and an alternative methodology for 
evaluating in real-time the parametric, reduced mass matrix M,,(u") is presented in 
Section 5.4.1. This alternative technique is particularly effective when the conditions 
(5.20) are not satisfied. 

Hence, attention is focused here on eliminating the second computational bottle- 
neck recalled above. This bottleneck is more significant than the first one because it 
arises even in the context of nonparametric problems, as long as they are nonlinear. 

To this end, let 


b, (unt H): p) = f, (u(t; H); B) - Sr(t; H) 
= W’ fy, (Vu, (t; W; p) -W' gy, (t; p) (5.21) 
ba, (Un(6 W); p) b,, (6) 
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denote the reduced, nonlinear, time-dependent, force balance vector. The com- 
putation of this nonlinear PROM quantity can be organized around the computa- 
tion of its two terms: that associated with the reduced, nonlinear, internal force 
vector b, (u, (t; }); p); and that associated with the reduced, external force vector 
b, (t; y). The potential for the evaluation online and in real-time of each of these 
two components of the reduced, nonlinear, force balance vector, and in the case of 
b;, (u„(t; u); p), that of its Jacobian with respect to u„(t; y), is discussed below. 


5.3.2.1 Real-time computation of the reduced nonlinear internal force vector and its 
Jacobian 


For many, but not all, partial differential equation (PDE)-based applications, the non- 
linearity in the FOM (5.1) is polynomial in Uy, Of degree d > 2. In this case, each i-th 


entry of the nonlinear, internal force vector can be written, for example, as follows: 


[fy, (Uw, (t; )s #)]; = Fi, uy, (6 W) 


Kaya i 
+ 2 (up, © G, uy, (t; W) (5.22) 
k=1 
KaD i 
+ (uy, (6 W) Gi, Wun, (t; H) F; „Duy, (t; H), 
k=1 


where i = 1,..., N}; [Q]; designates here and throughout the remainder of this chapter 
the entry in row i of the vector Q; F; 00 e RU: k = 0,..., [(d — 1)/2); G; (4) € 
RYM; and G; (uw) € RY, k =1,..., [(d - 1)/21. 

From the subspace approximation (5.17) and from (5.22), it follows that 


[fy,(Vun(G ws p)]; = F; g)Vu,(t; p) 


[(d=1)/2] a ‘ k 
+ (u, (6 WY G;,,(H) Vu, (ts p) (5.23) 
k=1 
[(d=1)/2] i 7 k 
+ (u(t; WV GW) Vu, H) Fi (DVU, (t; u) 
k=1 


is a polynomial function in u, of the same degree d. Next, from (5.21) and (5.23), it 
follows that 


Nn 

[By, (un (¢s W; 4)]; = [W' fy, (Vu, t; W; 1]; = DCW], Vans W) 
jel 

Nn [(d=1)/2] k 


+ IW] > (un WVG, Vu, (t W) 
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Nn [(d=1)/2] 


+ YW Yo (un DY G, Vu, (tw) E, (Vult W, 
jl k=1 


where [Q]; designates here and throughout the remainder of this chapter the entry 
in row i and column j of the matrix >. The above expression for the i-th entry of the 
reduced, nonlinear, internal force vector can be rewritten as 


Nn 
[b (u(t; W; 4)]; = [W" fy, (Vun (t; W; 1]; = X [W]; OV u(t; p) 


J=1 precomputable 


eR™ 

Nn [(d=1)/2] 7 . S 
+> > w(u H) V G; (WV u(t; w) (5.24) 

j=1 k=l precomputable 

eR™ 

Nn [(d=1)/2] j T s 
+ > > (u (t; WV Gj; (DV u(t; w) [W]; F; OV u(t; y). 

j=l k=1 precomputable precomputable 

eR™ eR™ 


This shows that each quantity arising in the evaluation of b, (u,(¢; 4); p) whose com- 
putational complexity scales with an integer power of the large dimension N, of the 
FOM can be precomputed offline for each queried parameter point, and contributes to 
the nonlinear PROM (5.6) a term that can be processed online, in a number of opera- 
tions that scales with an integer power of its small dimension n « Ny. 

Expression (5.24) also shows that for a low-order polynomial nonlinearity in the 
internal force vector — say, a quadratic nonlinearity (d = 2) - and a queried parameter 
point u* € P, b, (u,(t bu”); p“) can be computed online and in real-time. However, 
for polynomial nonlinearities of higher degrees (d > 2), the offline cost associated 
with precomputing the various projections identified in (5.24) becomes prohibitively 
expensive and the real-time evaluation of b, (u,(t; wu”); 4*) becomes less likely. 

Note that if the reduced, nonlinear, internal force vector b, (u,(¢; M); H) is a poly- 
nomial function of u, of degree d, the Jacobian of b, with respect to u, — and therefore 
the Jacobian of the reduced, nonlinear, force balance vector b,,(u,,(t; p); p) (5.21) with 
respect to u,, — is a polynomial function of u,, of degree d-1. For this reason, all conclu- 
sions stated above regarding the real-time evaluation of b, (u,(¢; 4); u) equally apply 
to the real-time evaluation of the Jacobian of b,, (u,,(t; 4); p) with respect to u,, - and 
therefore the Jacobian of the reduced, nonlinear, force balance vector b,,(u,(t; 4); 9) 
(5.21) with respect to u,,. 

The practical significance of the offline-online methodology highlighted in (5.24) 
for computing in real-time the reduced, nonlinear, internal force vector of a nonlinear 
PROM is underscored by its availability in many software products, including Vega, 
the physics-based computer graphics library for the simulation of the dynamics of 
three-dimensional deformable objects [8]. 
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Remark 5.2. In general, the reduced, nonlinear, internal force vector exhibits a low- 
order polynomial dependence on the vector of generalized coordinates u, if the non- 
linear, internal force vector exhibits a low-order polynomial dependence on the vec- 
tor of DOFs uy, . In practice, whether the latter holds true or not may depend on the 
choice of DOFs. For example in computational fluid dynamics, the discretization of 
the primitive form of the Navier-Stokes equations using primitive variables leads to 
a quadratic dependence of the nonlinear, internal force vector on these variables. On 
the other hand, the discretization of the conservative form of these equations using 
conservative variables cannot lead to such a dependence. 


Now, it is noteworthy to mention that if the external force vector is fixed in time, 
b, = b, (u) can be precomputed offline (see (5.21). In this case, the low-order poly- 
nomial dependence on the solution vector examplified in (5.22) - with d = 2 - de- 
fines the class of nonlinear FOMs for which, given a queried parameter point p* € P, 
the computational bottlenecks associated with the evaluation of the reduced, nonlin- 
ear, force balance vector (5.21) and/or its Jacobian can be eliminated using the exact, 
precomputation-based methodology described in this section. 


5.3.2.2 Real-time computation of the reduced time-dependent external force vector 


If the external force vector is time-dependent, b), = b, (t; u), and the real-time com- 
putation of this reduced vector requires special attention. 

To this effect, it is noted that in many, but not all, PDE-based applications, the 
time-dependent, external force vector can be divided into spatial and temporal com- 
ponents as follows: 


$y, (6 H) = BQ) am(t; u), where B € RY" am € R”, m < Np (5.25) 


B is a Boolean matrix that specifies the nonzero entries of Sn, (É; H), and a,,(t; p) is 
an amplitude vector that stores their time histories. In this case, the reduced, time- 
dependent, external force vector can be computed as follows: 


by (tw) = WB) an(ts p). 
precomputable 
eR™™ 


Hence, if 
Sn, (t; H) = B(H) am(t; H), Be RY", am € R”, and m « Np (5.26) 


where m < N, can be interpreted as m is of the order of n, b, (t; p) can be efficiently 
computed online and in real-time by precomputing offline its time-independent part 
Ww’ B(p). 
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5.3.2.3 Real-time computation of the reduced nonlinear force balance vector and its 
Jacobian 


Consider again the low-order polynomial dependence on the solution vector exam- 
plified in (5.22) — with d = 2 - and the spatial-temporal multiplicative decomposition 
conditions (5.25) and (5.26). Collectively, these conditions define the class of nonlinear 
FOMs for which, given a queried parameter point u* € P, the computational bottle- 
necks associated with the evaluation of the reduced, nonlinear, force balance vector 
(5.21) and/or its Jacobian can be eliminated using the exact, precomputation-based 
methodologies described in Sections 5.3.2.1 and 5.3.2.2. 


5.3.2.4 Real-time treatment of a parametric dependence in the reduced nonlinear 
force balance vector and its Jacobian 


There are two main cases to anticipate here: 

— Case 1: The interest is in a suite of nonlinear simulations, where the parameter 
point p € P is fixed within a simulation but varies across the simulations. 

— Case 2: The interest is in a single nonlinear simulation, where either or both of the 
following issues must be dealt with: 

— Case 2a: pis configuration-dependent and therefore may vary when the so- 
lution vector uy, (¢; 4) - and hence, the vector of generalized coordinates 
u,,(t; p), the nonlinear, internal force vector fy, (Vu,,(t; p); p), and the re- 
duced, nonlinear, force balance vector b,,(u,,(t; u); p) (5.21) — vary within 
the simulation. This is the case when, for example, the FOM (5.1) is associ- 
ated with a structural dynamics or solid mechanics problem, p contains one 
or several shape parameters, and the deformations are sufficiently large to 
change the shape of the computational domain at each time step. 

- Case 2b: pis varied in time — for example, in the external force vector gy, (t; p) 
and therefore in the reduced, external force vector g,,(t; 4) and the reduced, 
nonlinear, force balance vector b,,(u,,(t; y); p) (5.21). This is the case when, 
for example, the FOM (5.1) is associated again with a structural dynamics or 
solid mechanics problem, u contains one or several shape parameters, the de- 
formations are sufficiently large to change the shape of the computational do- 
main at each time step, and gy, (t; y) is a pressure-induced, time-dependent, 
external force vector. 


In Case 1, the exact, precomputation-based methodologies can be applied as de- 
scribed in Sections 5.3.2.1 and 5.3.2.2. In Case 2a, the parameter dependency in the 
reduced matrices [W],F) (WV, [W]jiF;,, (HV. V'G, (WV, and v'G,, (WY (see 
(5.24)) must be efficiently treated: This can be done using the method of interpolation 


on matrix manifolds described in Section 5.4.1. Similarly in Case 2b, the parameter 
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dependency in the reduced, external force vector can be efficiently treated using the 
interpolation method described in Section 5.4.1. 


5.4 Approximate reconstruction methodologies 


Within the inexact approach developed so far for eliminating the computational bot- 
tlenecks associated with the repeated constructions of the reduced quantities defining 
a PROM, two methodologies stand out from the rest. The first one specifically targets 
parametric, linear PROMs. The second methodology is equally applicable to paramet- 
ric and nonparametric, linear and nonlinear PROMs. 


5.4.1 Database of linear PROMs and interpolation on matrix 
manifolds 


For a sufficiently large and/or high-dimensional bounded parameter space P, and/or 
in the presence of large sensitivities of the parametric FOM with respect to p, the di- 
mension n of the global right ROB Y for which the subspace approximation (5.17) is suf- 
ficiently accurate may be too large to enable a PMOR to be “compact” and/or perform 
in real-time. Furthermore, if for a parametric, linear FOM the conditions (5.20) are not 
satisfied, the reconstruction in real-time of the PROM (5.19) at a queried but unsampled 
point u* € P may not be feasible. For these reasons, an alternative methodology has 
also been developed in the literature for eliminating the computational bottlenecks 
associated with the efficient processing of parametric, linear PROMs. 

Specifically, an alternative methodology was proposed in [1] for efficiently treat- 
ing the dependence of a linear PROM on a parameter vector. It consists in: generating 
offline a database of pointwise, linear PROMs; equipping it with a family of algorithms 
for interpolation on matrix manifolds; and applying these algorithms online to build 
in real-time a PROM at a queried but unsampled point u of the parameter space P. Sub- 
sequently, this alternative methodology was fully developed in [3] and demonstrated 
for realistic fluid, structure, and fluid—structure interaction problems. Recently, it was 
refined in [5]. In the context of the parametric, linear PROM (5.12) — which can be de- 
fined by the triplet (M,,(#), K, (H) y(t; W)) — this methodology for treating the param- 
eter dependency operates as follows: 

1. Offline (divide) 
- Apply an appropriate greedy procedure to sample n,, points u; in the parame- 

ter space P. 
- At each sampled point u; € P, i = 1,...,n,, apply any preferred technique 
such as, for example, the proper orthogonal decomposition (POD) method of 
snapshots [35] in the case of a linear Galerkin-PROM, or the balanced POD 
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2. 


method in the case of a linear Petrov-Galerkin PROM [40], to construct 
Mi, = M,(#;) = Wi My, V; = W” (u) My, DV) 
Ky =K, (H) = W7 Ky, V; = W" (a) Ky, UDV), 


(5.27) 
and 


g,(t) = S(t U) = Wi By, (t) = W"(H)gy, (6 p). 
Here, the notation W; = W(u;) (Y; = V(H,;)) specifies that the left (right) ROB 
is constructed at the point u; € P and that in this sense, it is a pointwise ROB. 
Store the set of precomputed reduced matrices (Mihe in a database Dm 
of pointwise, linear M,,-PROMs, the set of precomputed reduced matrices 
a ad in a counterpart database Dx, and the set of precomputed reduced 
vectors gio}, in a counterpart database Dg. 


Online (conquer) 


For each queried but unsampled point p* € P, construct M} = M,(u*), K} = 
K,(u"), and g} (t) = g,(t; p*) as follows: 

Identify a priori three matrix manifolds Mm, Mx, and Mg, on which M,, 
K,,, and g,(t) lie, respectively, by identifying the most important algebraic 
property characterizing each of these matrices as explained below (note that 
by default, any p x q real-valued matrix belongs to the manifold R?**). 
Interpolate in real-time the precomputed reduced matrices Mİ (5.27) on Mm, 
the precomputed reduced matrices Kİ (5.27) on Mx, and the precomputed 
reduced vectors gi (t) (5.27) on Mg. 


The first version of the methodology described above for treating efficiently the param- 
eter dependence of a linear PROM was developed in the context of linear/linearized 
Galerkin-PROMs [2]. It focused on: precomputing a set of pointwise right ROBs DAKA 
of the same dimension N, x n; storing these pointwise ROBs in a database Dy; and 
for each queried but unsampled point u* € P, interpolating the precomputed set 
of pointwise right ROBs to construct V,» = V(u"). The variant methodology out- 
lined above and first proposed in [1, 3] interpolates directly the pointwise PROMs 
{(Mi, Ki, gO (5.27) — instead of their underlying ROBs {V;, Wilt — in order to 
construct the pointwise PROM (Mj, K}, g7(t)) at u* € P. Hence, this variant offers the 
following added benefits: 


1. 


It replaces the interpolation requirement that all precomputed pointwise ROBs V; 
have the same large and small dimensions N, and n, respectively, by the relatively 
lesser requirement that all PROMs {(Mi, Ki, si), have the same dimension n. 
Indeed, while the former requirement restricts the construction of the pointwise 
FOMs {(My, . Ky, Sh, OAA to the same mesh or to topologically identical meshes 
in order to guarantee the same large dimension N, for each FOM (y,), the latter 
requirement allows these FOMs to have different dimensions (N,);. Therefore, it 
frees their construction on different meshes with different sizes [5]. 
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2. For each queried but unsampled point u* € P, it eliminates the need to form 
explicitly the matrix-matrix-matrix products WiMy, V, and WIKA, V after the 
right ROB Y, has been computed in real-time by interpolation of the set of point- 
wise, right ROBs (vit This is a critical advantage over the first version [2] based 
on the interpolation of ROBs, as the computational complexity of each of the 
aforementioned triple matrix product grows as © (Npn). 


Each of the reduced matrices M,, and K, and the reduced vector g„(t) can be charac- 
terized by identifying a priori the most relevant manifold it belongs to. For example, if 
My, is SPD and a Galerkin projection is chosen (W = Y), M, belongs to three different 
manifolds: the manifold of nxn real matrices, R™”; the manifold of invertible matrices 
of size n, GL (n); and the manifold of SPD matrices of size n, SPD (n). In this case, SPD 
(n) is the most relevant manifold as any matrix that lies on it is real-valued, of size n, 
and nonsingular. On the other hand, g,,(t) belongs to the manifold of nx1 matrices. The 
main objective of interpolation on a matrix manifold is to preserve such a character- 
ization during the interpolation process, so that at the queried but unsampled point 
py’, the interpolated PROM (M7, IK*, g*(t)) inherits this characterization. Otherwise, 
the standard (coefficient-by-coefficient) interpolation method does not guarantee this 
objective, except for the manifold R’“? — that is, for the case of g„(t; p) (p = n, q = 1), 
where the characterization g„(t; 4) € R” is not particularly specific. 

Let {Aly denote the set of matrices to be interpolated on a matrix manifold M 
on which they lie. For example: 
1. Al = Mİ = M = SPD (n), GL (n), or simply R™". 
X Al = Kİ = M = SPD (n), GL (n), or simply R™"”. 
3 A} =gi(t} > M =R”, 


Among these matrices, choose A’, as a reference point on M. Let G, denote an ele- 
ment of the tangent space 7,; M to M at A’, and let B, € M denote a point on M in 


a neighborhood of A’. Whichever matrix manifold M is chosen to support the inter- 
polation to be performed and whichever reduced matrix AŻ, is chosen as a reference 
point on this manifold, the set of matrices {aly can be interpolated on M using the 
method described in Algorithm 5.1 and graphically depicted in Figure 5.1. 
Essentially, Algorithm 5.1 starts by “moving” the set of matrices Al, PSs Ny 
buti + j, to the tangent space to the matrix manifold M at the chosen reference point 


A, Ty, M, using the logarithmic mapping Log al, (AŻ). This leads to the set of matrices 
Gi = G,(u;), i = 1,...,n,. Since Ty, M is a linear vector space, Algorithm 5.1 applies 
standard interpolation in this space to obtain the interpolated matrix G} = G,(u") 
at the queried but unsampled point u* € P. Next, it moves back G} to M using the 
exponential mapping Exp ai (G;) to deliver the desired matrix A% = A,(y"). 

Table 5.1 gives the expressions of the logarithmic and exponential mappings 
for each of the example matrix manifolds mentioned above. As shown in [3], these 
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Algorithm 5.1: Interpolation of {Aly on a matrix manifold M. 


Input: n, matrices An Sr A lying on M 
Output: Interpolated matrix A} = A,,(u") at a queried but unsampled point u* € P 


1: Choose j € 1,..., ny {the interpolation process occurs in the linear vector space 
Tai M} 

: fori=1,...,n, do 

Compute Gi, = Log al, (A!) 

: end for 

: Interpolate independently each entry of the matrices Gi, i=1,...,n 
obtain G7 = G,,(u*) 

6: Compute A; = Exp a, (G3) 


v FW NY 


u in order to 


G; = Log, :(A’) 


Figure 5.1: Interpolation of a set of matrices {al sie ona matrix manifold M. 


Table 5.1: Logarithm and exponential mappings for some matrix manifolds M. 


M RM GL (n) SPD (n) 


Logs, (Bn) By-A, log(ByA;') log(A,"/7B,A,"/”) 
Exp, (G) An+G  exp(G)A, A}? exp(G)An/? 


mappings and the entire Algorithm 5.1 can be implemented in real-time. For high- 
dimensional parameter spaces P, standard interpolation in 7,; M is perhaps most 
conveniently performed using radial basis functions [34]. 


Remark 5.3. Here, two comments are noteworthy. First, the systems of generalized 
coordinates underlying the linear PROMs stored in a database must be coherent, in 
order to enable the coherent interpolation of these PROMs. This requirement can be 
enforced offline, using the simple postprocessing algorithm described in [3]. Second, 
Algorithm 5.1 is not guaranteed in principle to preserve the numerical stability of the 
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interpolated PROMs. While this has never been observed to be an issue, the real-time 
stabilization algorithm described in [4] can be applied to the interpolated PROM to 
guarantee its numerical stability. 


5.4.2 Hyperreduction 


Finally, consider the most general case where any of the following conditions holds: 

- The parametric, nonlinear FOM (5.1) is not characterized by a low-order polyno- 
mial nonlinearity in the internal force vector fy, (Uy, (t; p); W9). 

- The parameter-affine representation (5.18) of the linear instance (5.11) of this high- 
dimensional computational model does not satisfy the conditions (5.20). 

- The parameter vector u characterizing the nonlinear FOM (5.1) or its linear in- 
stance (5.11) may vary within a single simulation for any of the reasons explained 
in Section 5.3.2.4, or any similar or related reason. 


In this case, the exact, precomputation-based methodologies described in Sections 
5.3.1 and 5.3.2 cannot be applied to eliminate the computational bottlenecks associ- 
ated with the repeated reconstructions of the reduced matrices or tangent matrices, 
and/or repeated evaluations of the reduced vectors that may arise during the process- 
ing of parametric, linear PROMs, and parametric or nonparametric, nonlinear PROMS. 
Furthermore, there does not seem to be a clear path that efficiently extends the inexact 
approach based on a database of pointwise, linear PROMs described in Section 5.4.1 to 
arbitrarily nonlinear problems. For all these reasons, another family of inexact meth- 
ods has been developed for addressing in the most general case outlined above the 
aforementioned computational bottlenecks. Here, these methods are collectively re- 
ferred to as hyperreduction methods — or the hyperreduction paradigm — even though 
their common underlying idea had emerged well before the word “hyperreduction” 
was coined in [33]. 

Hyperreduction methods mitigate or eliminate the computational bottlenecks as- 
sociated with the repeated reconstructions of projection-based reduced-order opera- 
tors by approximating these operators using a computational complexity that is inde- 
pendent of the dimension N, of the high-dimensional FOM. Hence, they trade some of 
the accuracy achieved by a PROM for speed. They can be classified in two categories: 
the approximate-then-project methods, which appeared first in the literature; and the 
project-then-approximate methods, which were developed more recently. 

As suggested by their label, approximate-then-project hyperreduction methods 
approximate first an operator of interest and then project the performed approxima- 
tion on the left ROB W (Petrov—Galerkin-PROM) or Y (Galerkin-PROM). Their under- 
lying common idea for avoiding a computational complexity that scales with the high 
dimension N, of the problem can be traced back to the gappy POD method [20]. This 
idea, which was originally developed for image reconstruction, can be described as 
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follows. First, the operator to be reduced — for example, in this case the nonlinear, 
internal force vector fy, (Vu, (t; 4); u) € R™ — is approximated using a small num- 
berm « N, of empirically derived basis functions, where m is not necessarily re- 
lated to the dimension n of the PROM to be hyperreduced. This can be written as 
fy, (Vu, (t; y); H) = fy, (Vu,(t; p); p) = UF,,(Vu,(t; p); p), where U € RN” stores in 
its columns the empirically derived basis functions and f,,(Wu,,(t; 4); 4) € R” denotes 
the vector of reduced or generalized coordinates of this approximation. This vector is 
computed such as to minimize the error of this approximation at a small number of 
computed rows of fy (Vu, (t; #); p) — that is, if Z represents the set of computed rows 
IZ] < Np feP'(Vu,(6 W; p) = arg min Ify, (Vu,(6 H); W) - UT fn (Vult; p); Dll = 
(U? J'EN, (Vu,,(t; p); p), where at designates the restriction of the vector or matrix 
A to its rows specified by the elements of 7; fy, € R; U? e R? pm, the super- 
script + designates the Moore-Penrose pseudo-inverse; and therefore (U7)* € R71, 
Note that if the number of empirical basis functions m is equal to |Z|, the optimal ap- 
proximation of fy, (Vu,,(¢; 4); 4) becomes fy, (Vu,(t; ); H) = ufoPt(yvu, (t; W; p) = 
U(U*) “fy, (Vun (ts p); p). 

Then, assuming the general case of a Petrov—Galerkin projection, the hyperre- 
duced, nonlinear, internal force vector is computed as £,(Vu, (ts W: H) = 
WÊ, (Yu, (t; u); H) = wTU(U? J'EN, (Vu,,(t; 4); p), where the tilde symbol applied 
to an algebraic quantity designates here and throughout the remainder of this chapter 
the hyperreduction of this quantity. The hyperreduced vector f (Vu,(t; H); p) is re- 
constructed each time t and/or u is varied by: precomputing offline the matrix-matrix 
product Q = wľu(uU? yt e€ RZ p and reconstructing online the approximation 
f (Vu, (t; H); H) = Qfy, (Vu, (6 H); p) in O(n|Z|) operations, where n « N, and 
|Z| « Np. 

The empirical interpolation method (EIM) introduced in [9, 26] — that is, almost a 
decade after the gappy POD method - and a variant of its discrete version known as the 
discrete EIM (DEIM) [15], as well as many other hyperreduction methods, including the 
missing point estimation approach [7], follow the same general idea, albeit for PDE- 
based applications. For this reason, the EIM is, unlike gappy POD, grounded in the 
continuum level. For elliptic problems, it enjoys some level of theoretical support. On 
the other hand, DEIM is arguably the most popular approximate-then-project hyper- 
reduction method to date, due perhaps to its black-box algebraic formulation. A ver- 
sion of this method tailored to FE approximations was recently described in [38] under 
the name “unassembled discrete empirical interpolation method” (UDEIM). Another 
noteworthy approximate-then-project hyperreduction method is the Gauss—Newton 
with approximated tensors (GNAT) method introduced in [12]. GNAT is also related to 
the gappy POD method. However, unlike EIM and DEIM, GNAT was conceived from 
the beginning for the Petrov—Galerkin rather than the Galerkin framework of model 
reduction. As such, it has a few distinctive features. More importantly, GNAT has been 
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successfully applied to the real-time solution of large-scale, three-dimensional, tur- 
bulent flow problems of industrial relevance [13]. 

On the other hand, project-then-approximate hyperreduction methods approxi- 
mate directly the projection on the left ROB W or V of an operator such as 
fy, (Vu, (t; p); H) € RX, They avoid a computational complexity that scales with 
N, by avoiding the construction in this case of fy,, and the matrix-vector product 
W’' fy, (Vu, (t; p); y) € R” or V' fy, (Vu,,(t; #); y) € IR". Among this family of hyper- 
reduction methods, the two most notable ones are the cubature-based approxima- 
tion method developed in [6] for computer graphics applications, and the energy- 
conserving sampling and weighting (ECSW) method developed in [22], characterized 
in [24] for second-order dynamical systems, and parallelized in [14]. Essentially, such 
methods proceed in two steps. First, they sample offline a given high-dimensional 
mesh to construct a reduced mesh, and attribute to each sampled element a weight- 
ing coefficient that they determine by solving a suitable optimization problem. Then, 
they approximate online each projection-based reduced-order operator to be recon- 
structed using a quadrature rule determined by the reduced mesh and its associated 
weights. For second-order dynamical systems such as those arising in wave propaga- 
tion, solid mechanics, and structural dynamics applications, ECSW is to date the only 
known project-then-approximate hyperreduction method with a provable structure- 
preserving property. For such problems, it specifically preserves the Lagrangian struc- 
ture associated with Hamilton’s principle. As such, ECSW can preserve the numerical 
stability properties of a discrete second-order dynamical system to which it is applied 
to, unlike the EIM, the DEIM, and GNAT. 

Hence, because they represent the state of the art of hyperreduction and have 
demonstrated success at enabling, for practical applications, the wall-clock time re- 
duction factors expected from PMOR, the remainder of this chapter focuses on the 
description and discussion of: the EIM, the DEIM, and GNAT within the category of 
approximate-then-project hyperreduction methods; and ECSW within the category of 
project-then-approximate counterpart methods. 


5.4.2.1 Approximate-then-project hyperreduction methods 


First, the hyperreduction of w-dependent vector quantities such as the external 
force vector gy, (t; p) is considered, followed next by the hyperreduction of non- 
linear, solution- and -dependent vector quantities such as the internal force vector 
fy, (Vu, (t; 4); #). In each case, the computation of the approximate function gy, (t; H) 
(and fy, (Vu, (ts H); )) is discussed, and then the efficient computation of the final 
result &„(t; u) (and £,(Vu,(t; 19; H)) is described. 
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5.4.2.1.1 Empirical interpolation method for -dependent functions 

The EIM was introduced in [9, 26] for the approximation at the continuum level of a 
family of parameter-dependent functions G = {g(-; u), p € P}c C°(Q), where in three 
dimensions, Q c R°. Here and throughout this section, the time dependence of a func- 
tion g € Gis accounted for by enlarging the dimension of P by 1 and treating time asa 
component of the parameter vector u. The method is built on basis functions obtained 
by sampling g at a suitably selected set of points in P, rather than on predefined ba- 
sis functions as in polynomial interpolation. Its purpose is to find approximations to 
elements of G through an operator Z% that interpolates the function g(-; u) at some 
carefully selected points in Q. Given a set of m basis functions {p,,...,9,,} that are 
linear combinations of m particular snapshots g(-; Mhm) ---- ZC; Mey), and a set of m 
interpolation points T, = {t',...,t'"} c Q to be properly selected — also referred to 
as magic points — the interpolant Tžg(; u) of g(; y), for y € P, admits a separable 
expansion and therefore can be written as follows: 


Trg H) => y (wp), x€Q. (5.28) 
j=l 


The -dependent coefficients y;(m) are obtained by fulfilling the interpolation con- 
straints 


TX 9(t'; p) = g(t}; p), i=1,...,m, (5.29) 


which yields the linear system of equations 
m . . 
$ y(w)p;(t") =g(t^; u), i=1,...,m. (5.30) 
j=1 


The matrix form of this system reads 
BmYH) =Sm(W), VH EP, 


where [Bm]; = P;(t’), [YOD]; = yj, and [8m]; = g(t’; u), fori,j =1,...,m. 

The construction of the basis functions p;(x), i = 1,...,m, yielding the approxima- 
tion space Xm = span{p;, .. . Pm}, is governed by a greedy procedure [29] that addition- 
ally yields the interpolation points T,,, = {t',...,t'} and a sample set of parameters 
ST {Mew PIA Mew all of which are needed for generating the basis functions. The 
greedy procedure is a two-step algorithm that can be described as follows: 

— Initialization step. The first sample point is chosen as 


1 7 
Heim = arg max|g (s Wl 
and therefore S, = {H4}. The first generating function is defined as 


&(x) = 8(% Mem): 
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The first interpolation point is selected as 


t' = arg max |é,(x)| 
xeQ 


and therefore T, = {t1}. Hence, the first basis function is constructed as 


Py (x) = &(x)/é,(t’) 


and therefore X, = span{p,}. Finally, the initial interpolation matrix is defined as 


[Bm] = p(t’) =1. 


At this stage, the available information allows the definition of the interpolant 
as the only function that is collinear with p, and coincides with g at t! — that is, 
Tkg(x; p) = g(t’; pp). 

Recursive step. At each j-th step, j = 1,...,m — 1, given the nested set of interpola- 
tion points T; = eee t) and the set of basis functions {p;,...,p;}, the G + 1)-th 
generating function is selected as the snapshot that is worst approximated by the 
current interpolant — that is, the snapshot that maximizes the difference between 
gand Tis. This can be written as 


Hii = arg max|g(s w -T786 Mi» ay (5.31) 
which yields the generating function 
gjr1(X) = g(x; BER 


and the set Sj,, = Sj U gii- The (j + 1)-th interpolation point is selected by first 
solving the linear system 


=. 


Y alt’)yi =at) i=j 


I=1 
in order to characterize the interpolant Z*é;,, (5.28), then evaluating the residual 
r1) = Gui) - GG), (5.32) 
and finally choosing the point in Q where ;,, is worst approximated — that is, 


t! = arg max Ir;.1(X)|- (5.33) 
xeQ 


This point selection is followed by the update Tj}, = Tj U {t1}. Then, the new 
basis function is constructed as 
§j41(X) = TPR) rju) 


i (X) = — = - 
Pj+t & (ti?) ZR Tja (t71) rj(ti*1) 


and therefore X;,, = span{p;}, i=1,...,j+1. 
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The entire procedure described above is repeated until a given tolerance £p is 
reached for the maximum norm of the residual (5.32), or until a given maximum 
number max of terms is calculated. 

EIM yields a sequence of hierarchical spaces X} c X, C --- C Xm and a set 
{P1,--->Pm} Of linearly independent basis functions. Moreover, the m interpolation 
points where the approximation is required to match the function being interpolated 
are iteratively determined in an adaptive fashion — that is, without having to recom- 
pute all previously selected points. 

An a priori error estimation, bounded in terms of the best approximation error, 
holds for EIM. Precisely, it is first noted that for any given m 


Tag p) = X g(t WL"), where [7"(x) = X oB) 


i=1 j=1 


and by definition, Mx )=6 
satisfies 


j» b j = 1,...,m. Then for any g € G, the interpolation error 


|86 W -Tag Dlr < (1+ Am) inf [86 W -Snl 
where 


m 
Am = sup È IG Œ) 


XEQ j=4 


is the Lebesgue constant (see, e. g., [29]) and {I” € Xm} denotes a set of characteristic 

Lagrangian functions (Er (x! ) = 6, i, j = 1,...,m). Here A,, depends on X,, and on the 

magic points Tm, but is w-independent. A pessimistic upper bound for the Lebesgue 

constant is Am < 2” —1. A posteriori estimates for the interpolation error can be found 
in [9, 19], and a link between the convergence rate of the EIM approximation and the 

Kolmogorov n-width of the manifold G is discussed in [29]. 

In practice, finding the supremum in (5.31) and (5.33) is not computationally fea- 
sible unless approximations of both Q and P are considered. Two noteworthy approx- 
imations are: 

- A fine sample SE, c P of cardinality |EN | = Nain to train EIM. 

- A discrete approximation Q, = ay , of Q of dimension nx. For example in an 
FE context, the points x* can be the vertices of the computational mesh, or the 
quadrature points of its elements. This approximation leads to N, DOFs, where 
N, = N}(ny) is the dimension of the FOM (5.1) and typically N} > nx. 


In this setting, a computable, algebraic version of EIM can be provided after the fol- 
lowing quantities are introduced: 
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- Avector representation gy, (M) € R` of g(-; H), with entries defined by 


[8n 0D]; = g0”; u), i= 1... Np 


where map is the function that maps each DOF of the FOM (5.1) to the vertex of Q, 
to which it is attached. This representation is obtained by evaluating the function 
gin Q, for anype P. 

- The matrix U € R%**" defined as 


U=[P]--- Pm] (5.34) 


whose columns encode the discrete representation of the basis functions {p,..., 
Pmb ive. [U]; = p;(x™?). 

- Asetofm interpolation indices Z = {i,,...,i,,} associated with n, < m inter- 
polation points {t!,...,t"} such that {t!,...,t”} = {x,...,xm}, where for k = 
1,..., Nm» J, is related to an interpolation index i, € Z via ją = map(i)). 


From (5.28), (5.30), and (5.34), it follows that the discrete representation of the inter- 
polation operator 7%, denoted by Ên, (u) € R^», is given by 


Sy, H) = UZm (H), (5.35) 


where g,,(H) € R” is the solution of the linear system 


EAN = ¥ [sn] [Pl = [8n, QO], l= b.m. (5.36) 
j=l 


Denoting by EN, (u) € R” the vector whose components are gx, (MI), = [8n,Q0];, for 
l = 1,...,m and noting that the m x m matrix B,, is easily formed by restricting the 
N, x m matrix U to the rows indexed in 7, i.e., Bn = UŽ, (5.29) can be written in 


compact form as 


U” gm(H) = Sy, (D. (5.37) 


From (5.35) and (5.37), it follows that 
Èn (D) = U(U*) ‘gy (H) VH €P, (5.38) 


which completes the description of the algebraic version of the EIM. A correspond- 
ing algorithm is given in Algorithm 5.2. Note that the solution of the dense linear sys- 
tem (5.37) requires O(m?) operations, thanks to the lower triangular structure of U7. 
Note also that at each iteration, Algorithm 5.2 involves the evaluation of gy (4) for 
p < Ef. Should this operation be expensive, one may form and store once for all the 


(possibly dense) matrix 


S= [ Sy, (u') | ide | Sy, (p=) ] e RN "rain 
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Algorithm 5.2: EIM (computable version): offline and online phases. 
Offline phase 
Input: Shae Qr, Mmax> EEIM 
Output: U, 7 
Lo 
tM = arg max ISu, Hho 


2: f = Sy, g) 
: 1 

3: 4 = m i 
4 = arg ‘a. [$n (Hil 


E Pa = Sn, WOEN HD, 
: Uc [pı], Z — {i}, T = {x"} 
: for k = 2 t0 Mmax do 
k_ T -1 T 
H =a max l8v, %0) - UCU) gy, Pleo 


eS train 


Wy DAu A 


if Igy, (H) - U(U7) "gy, (Hoo < Erm then 
break 
10: endif 
Ut, = Sy, H") - UU) gi, O) 
12: i, = arg nna l[tiil 
13: Py = Yk/ [Ekli 
w: Ue (U p}, Te Zu fi}, T T uU i} 
15: end for 
Online phase 
Input: py", Tp > ut 
Output: g,,(H") 
1: Form ZN, (yu) by evaluating g(; uw”) at the interpolation points Tan = (act) 
2: Solve U Sm(H") = By, (H") 


before entering the while loop. However, already for moderately large values of N, and 
Ntrains Storing the matrix S can be quite challenging. 

After the approximation (5.38) has been computed, the final computation of 
the hyperreduced quantity g,(u) = W’ gy, (u) (or g&u) = vgn, (H) in the case 
of a Galerkin projection) is performed as follows. In the offline phase, the matrix 
Q = W'U(U7)? € R®™ is precomputed. Then, for each queried parameter point 
p“ encountered in the online phase, g,(u*) = Qsn, (u*) is computed using a matrix- 
vector product whose computational complexity is only O(nm). 


Remark 5.4. Note that if, for whatever reason, a set of linearly independent basis func- 
tions {p,,...,P,} were given, then the procedure of finding the interpolation points (or 
magic points) is well-defined. This remark underlies the idea exploited by DEIM, as al- 
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ready highlighted in [29] in a general setting. While the computable, algebraic version 
of EIM given in Algorithm 5.2 was never published before — and certainly not before 
the introduction of DEIM in [15] — it can be reasonably assumed that this algebraic ver- 
sion or some variant of it (for example, see [10]) was used to compute the numerical 
results reported in [26] and in subsequent papers [11, 25]. 


5.4.2.1.2 Discrete empirical interpolation method for u-dependent functions 
Introduced in [15], the DEIM can be described as a variant of Algorithm 5.2 in which the 
construction of the basis U is not necessarily embedded in the greedy procedure used 
for guiding the remainder of the computation of the approximation gy, (#). Specifi- 
cally, DEIM approximates a function gy, (H) € IR» similarly to EIM, by projection onto 
a low-dimensional subspace spanned by a basis U. This can be written as 


Sy, (H) = $y, (H) = UZm H), 


where U = [p,|... [Pm] € RN", Zm(H) € R” is the corresponding vector of general- 


ized coordinates, and m « N,. DEIM also selects the interpolation points iteratively, 
using the same greedy procedure as EIM. However, DEIM was introduced in [15] by 
proposing to construct U through the application of POD to a set of computed snap- 
shots 


S= [ Sy, (HDem) | ves I Sn, Hirm) b Ns > m. 


Hence, the key descriptors of this method can be summarized as follows: 
1. Construction of a set of snapshots obtained by sampling gy, (4) at values Uppy, 
i=1,...,n,, and application of POD to extract the basis 


U = [P1] ... [Pm] = POD([ gw, (Hbem) | --- 181,(HDEm) b Erop) (5.39) 


where €pop is the usual prescribed tolerance for basis truncation. 

2. Iterative selection of the set of m indices Z c {1,..., Np}, where |Z| is therefore 
equal to the dimension m of the basis U, using a greedy procedure and this basis. 
This operation, which minimizes at each step the interpolation error measured in 
the maximum norm over the set of aforementioned snapshots, is the same as the 
selection of the magic points in EIM. 

3. Computation of the vector of generalized coordinates g,,(u), given a new y. This 
computation is performed by imposing interpolation constraints at the m points 
of T corresponding to the selected indices of gy, (4). It requires the solution of the 
following linear system: 


UŽ gm) = gN, (D, (5.40) 


where, as previously, U7 € R®™ and Sy, (4) € R” are the matrix and vector 
formed by the rows of U and gy, (4) indexed in Z, respectively. As a result, 


Sy, (H) = U(U*) ‘gy, (5.41) 
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Algorithm 5.3: DEIM: offline and online phases. 
Offline phase 
Input: S, €pop 
Output: U, 7 
1: [P1]... [Pm] = POD(S, epop) 
2 i = ca pa Ilp1];l 
: Uc [p], Z - {i} 
: for k = 2 to m do 
rk = Px- UUT) pK 
ix = arg ues [Jil 
7z Ue [(Up,|,2—TLutigt 
8: end for 


Online phase 
Input: p*, Z, U? 
Output: g (u`) 
1: Form ZN, (p*) by evaluating gy, (u*) at the interpolation indices Z 


2: Solve U Zm(H*) = Sy, H”) 


The construction of the basis U and the selection of the set of indices Z are summa- 
rized in Algorithm 5.3. The well-posedness of the general, dense, linear system (5.40) 
is amply discussed in [15] and related papers. The computational complexity of its 
solution is O(m? ). 

The approximation error gy, () - Ên, (u) can be bounded as 


Igu,@0) - Èn, Dll < (U7) “Ip I- UU") gy, Wl (5.42) 


where 
(1 - UU") gy, D2 = Omar (5.43) 


and 0; is the first discarded singular value of the matrix $ when applying the POD 
procedure to construct U € R”, The result (5.43) holds for any H € P, provided that 
a suitable sampling of the parameter space has been carried out to build the snapshot 
matrix S. In that case, the predictive projection error (5.43) is comparable to the train- 
ing projection error 0m1- The estimate (5.42) is built on the information related to the 
first discarded term: It can be seen as a heuristic measure of the DEIM error. Further re- 
sults about a posteriori error estimation for POD-DEIM reduced nonlinear dynamical 
systems can be found in [41]. 

After the approximation (5.41) has been computed, the final computation of the 
hyperreduced quantity g,(u) = W’ gy, (u) (or &, (9) = V’ én, (p) in the case of a 
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Galerkin projection) is performed as previously described for the case of the EIM 
method, in O(nm) computations only. 


5.4.2.1.3 EIM and DEIM for solution- and -dependent functions 
The computable, algebraic version of either the EIM or the DEIM can be equally used 
to speed up the evaluation of the parametric, reduced, nonlinear, internal force vec- 
tor f,,(u,,(t; p) defined in (5.8). Because of space limitations, however, only the case of 
DEIM and the context of the Galerkin projection are considered here. As far as hyper- 
reduction is concerned, similar conclusions can be drawn for the EIM and the more 
general context of the Petrov-Galerkin projection. 

The problem is to find m (t, w)-independent basis functions Pj £ R™, j=1,...,m, 
and a (t, y)-dependent vector f „(Vu,(t; y); 4) € R”, such that the nonlinear, internal 
force vector can be approximated as 


fy, (Vun (6 W; H) ~ Îy, (Vun (ts W); H) = Usf (Vu, (ts p): p), 


where U; = [P1 | --- | Pm € R~*™, This problem can be solved as follows. 
In an offline stage, DEIM is applied to a set of snapshots 


S; = {fy, (Vu, (t5 p°) p°), S= 1,....ng k=0,...,N,-1} (5.44) 
in order to build the basis U; and a set of m interpolation indices Z, = {i}/%,, 
where |Z-| = m is the number of empirical basis functions. For any new p € P, 


f,,(Vu,,(6;);H) € R” is then obtained during an online stage by solving, similarly 
to (5.36), the following linear system of equations: 


m 


[fy, (Vu, (t; WD: u)]; = X [En (Vun (t; W; u)] lp; = [fy,(Vun(6 1], 
j=1 


forl =1,...,m — that is, 
fy, (Vun(t W:M) = U;(UF) "fy, (Vu (ssp) VH €P. 


As before, UF and fy, (Vu,,(t; u); u) denote here the matrix and vector formed by the 
T rows of Us and fy, (Vu,,(t;); 4), respectively. Then, the parametric, reduced, non- 
linear, internal force vector f,,(u,,(t; y) = V' fy, (Vu,,(t; 4); 4) can be approximated as 
follows: 


fa (un(t; Hs) ~ Fy(Un(ts W: 4) = V" fy, (Vunt: W; p) 
= V'U;(UF) fy, (Vut; p); p). 


nxm mx1 
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Note that fy, (Vu,,(t; u); u) can be efficiently evaluated by employing on the reduced 
mesh associated with the selected interpolation indices the same assembly rou- 
tine used in the context of the full-order problem: This is further described in Sec- 
tion 5.4.2.1.5. 

A key fact is that the snapshots of the internal force vector highlighted in (5.44) 
depend on the PROM-based approximation of the state variable. This implies that 
in principle, a nonhyperreduced PROM must be first built to provide the snapshots 
vu, (t; p5), s =1,...,n,, k = 0,...,N; — 1, which is computationally inefficient. De- 
pending on the application however, this issue can be avoided and the following snap- 
shots of the internal force vector can be precomputed instead: 


S; = {fy, (uy, (t; p°); p°), s=1,...,n k=0,...,N;-1}. (5.45) 


Such snapshots can be evaluated while processing the FOM and computing solution 
snapshots of the state variable in order to construct the PROM. If there exists a K > 0 
independent of N, such that 


lv, (uy, (64s p) - fy, (Vun (ts W; p)|| < Klun, (GH) - Vu, (6 | 


for every t and p, then the error due to using the snapshots (5.45) instead of their coun- 
terparts (5.44) for performing the hyperreduction of the internal force vector can be 
kept under control. 

The DEIM can also be used to compute an approximation of the reduced Jacobian 
matrix K,,(u,,(t; p); p) defined in (5.9). Indeed, 


K,(un(t H); p) ~ Kn (uns H); p) 


_ ofn N 
= au, ons H); H) 


= V'Ry,V 
T Trl. TF I 
= VU (UZ) KE (Vu, (tsa 9V7, 
nxm mxn 
where the p-independent quantity vuU 
Ky, (Vu,,(t; y); H)V7 € R™" can be assembled online. 


)? can be precomputed, and 


5.4.2.1.4 Gauss—Newton with approximated tensors method 

Whereas the EIM and DEIM were developed in the context of the Galerkin-PMOR, 
GNAT was introduced in [12] in the Petrov-Galerkin context. Like the DEIM, it was 
conceived to operate at the fully discrete level. However unlike the DEIM and the com- 
putable version of the EIM, it was designed to operate on discrete PROMs of the form 


W'ty, (Vu, (t's u); p) = 0, (5.46) 
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where tt! denotes the time instance at the (k + 1)-th computational time step; 


YN, (Vu, (t ae p); H) = M, ù, (t a p) + f,(un(t sae p): H) - ge; p) (5.47) 


is the residual associated with the discretization of the FOM (5.1); and the left ROB 
W is chosen such that the solution of the projected nonlinear system of equations 
(5.46) results in the minimization in the two-norm of the discrete, nonlinear residual 
YN, (Vu,(¢ k+l, H); p) over the approximation subspace associated with the right ROB 
Y. Namely, GNAT operates in the context of an iteration-dependent left ROB 


> H); WY, (5.48) 


where Jy, (vu, (t*"; H); p) = M (Yu, (t k+l. u); u) is the Jacobian matrix of the 
h 


discrete, nonlinear residual ry,(Vu,(t kit, }); H). In this case, solving the system 


of equations (5.46) using Newton’s method is equivalent to solving the nonlinear, 
least-squares minimization problem 


krt, H) = arg min |[ry, (Vx; W)ll2 (5.49) 


xeR" 


u,(t 


using the Gauss—Newton method. For this reason, the projection method summa- 
rized by (5.46) and (5.48) is commonly referred to in the literature as the least-squares 
Petrov—Galerkin (LSPG) projection method [12]. 

For a self-adjoint FOM (5.1) characterized at the semi-discrete level by an SPD Ja- 
cobian matrix Jy, (Vu,(t k+1, u); u), the Galerkin-PMOR (W = V) approach can also be 
shown to minimize the discrete, nonlinear residual (5.47), but in the Jy, norm. How- 
ever, in the general case where the Jacobian matrix Jy, (Vu, (t; H); p) is not SPD - 
for example, when the FOM (5.1) results from the semi-discretization of the Navier— 
Stokes equations — the Galerkin-PMOR approach lacks the optimality property of the 
LSPG approach associated with the aforementioned minimization process. 

The solution of the LSPG minimization problem (5.49) by the Gauss—Newton 
method incurs the solution of a sequence of linear, least-squares problems of the form 


min Jy, (Vul?; pw) Vx + ty, (Vu; pI), (5.50) 
where p denotes the p-th iteration of the Gauss-Newton method. Thus, even though 
the dimension of the search subspace associated with the solution of the problem 
(5.50) isn « Nj, the computational complexity of any processing of the LSPG-PROM 
defined by (5.46) and (5.48) remains dependent on the dimension N, of the origi- 
nal FOM (5.1). To address this issue, GNAT approximates the columns of the high- 
dimensional matrix-vector product In, (Vu?) (t et H); WV and the high-dimensional 
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nonlinear residual ry, (Vu Cae H); H) by projection onto the low-dimensional sub- 


spaces spanned by two bases to be determined, U; € RN and U, € RN*™, 
respectively, where my « N, and m, < N,. This can be written as 


Jy, VUP (E; W; WV = Ty, VVP ws p) = UTP, 


= (5.51) 
ry, (Vu? (t; w; p) = Fy, (Vue; p); p) = UR), 


where Tn) e€ R™*" and ro (p) € IR™. Specifically, GNAT constructs each of the 
bases Uy and U, by computing snapshots of the matrix-vector products incurred by 
IN, (Vu (t krl, H); WV and the nonlinear residual tn, (Vu) (t er, H); p), respectively, 
and compressing them using the POD method. 

As in EIM and DEIM, GNAT computes the reduced coordinates associated with the 
subspace approximations (5.51) by minimizing the interpolation error at a selected set 
of interpolation indices Z determined by a greedy procedure. Specifically, GNAT uses 
for this purpose a gappy POD reconstruction technique [20, 13] where the interpolation 
error is minimized at a selected set of indices Z in the least-squares sense. This yields 

J% (WD = arg min |Jg (Vuy (t5 p); pV - Uy xlr 
xER”D "Y 
= (UF) "Ty, Vu (tS p); WV, 


pa (5.52) 


r® (p) = arg min lrg (Vue; w; p) - Uy xl 
xeR™ 


= (Uy) rg, (Val? (t p); W, 


where the superscript Z designates, as before, the restriction of an algebraic quan- 
tity to its rows associated with the indices in Z, and the superscript + designates the 
Moore-Penrose pseudo-inverse. Unlike in the EIM and DEIM, where |Z] is chosen to 
be equal to the dimension m of the function to be approximated for the purpose of 
hyperreduction, |Z| is governed in GNAT by the more general constraints |Z| > m, and 
|Z| > my. These constraints suffice to make the interpolation problems well-posed. 
However, it is noted here that: both EIM and DEIM can be modified, if desired, to use 
a set of indices Z whose cardinality is larger than m; and the reconstruction in GNAT 
can be constrained to the case where |Z| = m, = my, if this is deemed more practical. 

Substituting the approximations defined by (5.51) and (5.52) into the linear, least- 
squares minimization problem (5.50) to be solved at each p-th Gauss—Newton iteration 
and exploiting the orthogonality property uf U = In, transforms this problem into 
the hyperreduced, linear, least-squares minimization problem 


min|(U;)"Iy, (Vu; w; pyx 
xeR” 


+ UUU ry, Vu? (t; u); Wl» (5.53) 


where the matrices Qy = (UF JË € R®*Z and Q, = uF U,(U,)* € R™*'7! can be pre- 
computed offline. The offline-online decomposition achievable here is the direct result 
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of the gappy POD-based approximation followed by the projection onto the left ROB W 
of full-order quantities (in this case, In, (Vu?) (t, H): p)V andry, (Vu (t, 1; W9), 
in the same fashion as was achieved for EIM and DEIM. Note that given my > n, the 
problem (5.53) has a unique solution. 

The above description of GNAT shows that the computational complexity of this 
hyperreduction method is independent of the dimension N, of the FOM (5.1). At each 
p-th Gauss—Newton iteration, this complexity has two parts. The first part, which is as- 
sociated with the computation of the matrix-matrix product QJ N, (vu (t k+1, W; WV 
and that of the matrix-vector product Qty, (Vut (t Krl, H); p), is O((myn+my)|Z|). The 
second part, which is associated with the solution of the linear, least-squares problem 
(5.53) of dimension m,xn, has the computational complexity omn?) when performed 
by forming and solving directly the normal equations associated with (5.53), or using 
instead a QR factorization-based approach. 


5.4.2.1.5 Mesh sampling 
The EIM, the DEIM, and GNAT have in common a procedure for constructing a set of 
indices Z. Collectively, these indices constitute in general a sampling of the DOFs of 
the FOM (5.1) that defines the vectors ZN, (t; p) and fy, (Vu, (t; u); y), or the vectors 
ty, (vu (t**1; u); p) and Jacobian matrices Vy, (vul(t**1; u); u), and leads to effi- 
cient hyperreduction. In most cases, the construction of these sampled vectors and 
matrices does not require access neither to the complete mesh nor to the full-order so- 
lution Vu,,(¢t; u). Instead, it requires access only to a small subset of the geometrical 
entities (for example, the elements of an FE mesh) related to the indices Z, and to the 
contributions of these entities to the aforementioned sampled vectors and matrices. 
For this reason, the concept of a reduced mesh was introduced in [13] in order to ease 
the implementation in practice of hyperreduction methods of the approximate-then- 
project type. In this concept, the reduced mesh is defined as the subset of the original 
mesh which contains only those geometrical entities that are essential to the correct 
computation of the aforementioned sampled vectors and matrices. Hence, depending 
on the type of the spatial discretization (i.e., FE, cell-centered or vertex-based finite 
volume [FV], or finite difference [FD]), the reduced mesh may represent a larger mask- 
ing of the full-order solution vector described by the larger set of indices Z, (|Z,| > |Z\). 
Specifically, it is fully described by Z and the stencil of the semi-discretization under- 
lying the FOM (5.1). As such, the reduced mesh describable by Z, enables the reuse 
of the same computational framework and corresponding software employed to con- 
struct the FOM (5.1), not only to efficiently compute the sampled quantities ZN, (t; p) 
and fy, (Vu,,(t; p); p), or ty, (Vu ct; wW; H) and Jy, (vu (th, }); p), but also to 
hyperreduce the PROM (5.6) and process the resulting hyperreduced PROM. 

The concept of a reduced mesh described above is illustrated in Figure 5.2, fora 
two-dimensional, first-order, vertex-based, FV semi-discretization based on triangu- 
lar elements. The left part of this figure shows the computational mesh with the dual 
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original mesh reduced mesh 


Figure 5.2: Illustration of the reduced mesh concept for a two-dimensional, first-order, vertex-based 
FV semi-discretization: original mesh (left), basic and enlargened sets of indices (middle), and re- 
duced mesh (right). 


cells (or control volumes) delineated using dashed lines. The middle part of this fig- 
ure highlights a sampled set of indices Z and the associated larger set of indices Z}. 
The right part of Figure 5.2 shows the reduced mesh associated with 7,. (Note that for 
an FE semi-discretization using the same mesh, the same set of indices Z leads to the 
same reduced mesh.) 


5.4.2.2 Project-then-approximate hyperreduction methods 


Unlike their approximate-then-project counterparts, project-then-approximate hyper- 
reduction methods approximate directly reduced quantities, such as the reduced, 
nonlinear, internal force vector f,(u,,(t; M); HW) = W’' fy, (Vu,,(t; p); p) or its Jacobian 
K,(u,(6 4); p) = W’ Ky, (Vu, (¢ W9); WV (6.9). Furthermore, unlike the EIM and the 
DEIM, the method ECSW overviewed below hyperreduces -dependent functions 
such as the reduced, external force vector g,,(t; y) and solution- and -dependent 
functions such as f,,(u,,(¢; 4); #) in the same manner. For this reason, the focus is set 
here on the general case examplified by the reduced, nonlinear, force balance vec- 
tor b,,(u,(t p); p) = f,(u,(ts w); H) - g(t; p) (5.21) and its Jacobian with respect to 
u(t; y), K,(uy(t 4); p). 


5.4.2.2.1 Energy-conserving sampling and weighting method 

The hyperreduction method ECSW can be derived and/or interpreted using two differ- 
ent but related approaches. Both are combined below to describe this method in the 
clearest possible manner, in the context of the FE, cell-centered or vertex-based FV, or 
FD method. 

Let € = {e1, e2, . . . , €n,} denote: the set of n, elements of a given mesh if the context 
is set to that of an FE or cell-centered FV semi-discretization over this mesh; the set of 
dual cells if the context is set to that of a vertex-based FV semi-discretization; or the 
set of nodes of the mesh if the context is set to that of an FD semi-discretization. 
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In the case of an FE semi-discretization, the evaluation of b,,(u,,(t; 4); y) and the 
construction of K,,(u,,(t; 4); p) are typically performed as described by the following 
equations: 


b, (u(t W; p) = 2 W'L2b,(LeVu, (ts p); H), (5.54) 
ecE 

K,(un(ts W; 4) = X WL K:(LeVu,(t; ws H)LeV, (5.55) 
ecE 


where L, is the d, x N, Boolean matrix that localizes a high-dimensional matrix de- 
fined over the entire mesh to the DOFs associated with the entity (element or dual cell) 
e; dẹ denotes the number of DOFs associated with this entity; b,(L,Vu,(t; 4); H) € 
RË denotes the contribution of this entity to the reduced vector b,,(u,,(t; u); p); and 
K,(L,Vu,,(t; p); H) € IR%*“ denotes the contribution of this entity to the reduced ma- 
trix K,,(u,(t; 4); y). 

In the case of an FV or FD semi-discretization, the evaluation and construction of 
the above reduced quantities are performed similarly — that is, 


b,(u,(t; W9; p) = Z W'Le be(Le, Vun (t; p); H), (5.56) 
ec€ 

K,(un(6s H); p) = * WL? K, (Le, Vu, (ts H); H)Le+ V, (5.57) 
ec€ 


where b,(L,, Vu,(t; u); p) and K,(L,,Vu,,(t; u); y) denote, as before, the contribu- 
tion of the entity e to the reduced vector b,,(u,,(t; u); p) and its contribution to the re- 
duced matrix K,,(u,(t; 4); H), respectively. However, each of these two contributions 
may depend in this case on the DOFs associated not only with the entity e, but also 
with neighboring entities as dictated by the semi-discretization stencil. This is desig- 
nated in (5.56) and (5.57) by the symbol + next to the subscript e of the Boolean matrix 
Le, of dimension (d,n,,) x n, where ne, denotes the number of entities participating 
in the evaluation of b,(L,, Vu, (t; 4); p) and K,(L,, Vu,(t; 4); 4), as required by the 
stencil of the chosen spatial discretization. 

To unify the notation adopted above for an FE, FV, or FD semi-discretization, no 
distinction is made in the remainder of this section between L, and L,,. Instead, it is 
assumed that if the spatial discretization is of the FV or FD type, L, is to be understood 
as Le,. 

Consider the variational setting underlying the construction of the nonlinear, 
force balance vector by, (Vu,(t; y); y) = fy,(Vu,(t; u); H) - Zn,(t; p) and tangent 
stiffness matrix Ky, (Vu,,(¢; 4); y). In this setting, W can be interpreted as a matrix 
of test functions and each entry [b,,];(u,,(t; 4); H) of the reduced vector b,,(u,(t; p); p) 
(5.54) can be interpreted as the virtual work of by, (Vu,(¢; p); p) along the virtual 
“displacement” [WT]; 

Now, let E = {€,6,..., @n,<<n,} C E represent a reduced mesh obtained by sam- 
pling the elements or dual cells of the mesh represented by €, as appropriate. The con- 
servation on this mesh of each virtual work represented by each entry of the reduced 
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vector b,,(u,,(t; 4); p) can be globally expressed as 


Y Web (LVun(ts ws w= È E&W Li be(LeVun(t; p); 4), (5.58) 
ecE ecEc€ 
teeta By (uy (52050) 


where the real-valued coefficients &* must be introduced so that the above equality 
may be feasible. 

Similarly, each entry [IK,,],(u,(t; 4); p) of the reduced matrix K„(u,„(t; u); H) (5.55) 
can be interpreted as the virtual work of the internal force vector Ky, (Vu,(¢ 4); 
WIVI] H along the virtual “displacement” [WT]; The conservation on the reduced 
mesh represented by € of this virtual work fori = 1,...,n andj = 1,...,n can be 
globally expressed as 


Y WL K,(LeVu,(t; u); OLV= X E W L Ke(LeVu,lt; u): H)LeV, (5.59) 
ecE ecEcE 
K, (u, (64); 4) 


K,(u, (603) 


where € and its associated set of real-valued coefficients ts are in principle 
the same as those that appear in (5.58), because Ky, (Vu,,(t; u); p) is the Jacobian of 
(fy, (Vu,(t)) - $y, (t)). In practice, these coefficients must be determined numerically 
such that both identities (5.58) and (5.59) hold approximately. For this reason, these 


identities should be rewritten as 


b,(u,(6 p); p) = b,(u, (6 p): p) = by E"W'L¢b,(L.Vu,(t p); H), (5.60) 
ecEcE 
K„(u,(t; p); p) ~ K, (u(t: p): H) = > E"W'LK,(LeVuy (tM); H) LV. (5.61) 
ecEcE 


Expressions (5.60) and (5.61) suggest that each of the approximations b,,(u,,(t; W); 
H) and K„(u,„(t; H); p) can be interpreted as a generalized quadrature rule, where the 
elements in € are the quadrature points of this rule and the real coefficients CAUA 
are its weights. 

If the high-dimensional matrix Ky, (Vu,(¢; 4); 4) is SPD and W = Y (Galerkin 
projection), or for some other reason the matrix product W'Ky, (Vu, (t; u); p)Y is SPD, 
then K,,(u,,(t; 4); p) is also SPD. In this case, K,(u, (6 H); p) should also be SPD, and 
therefore each coefficient ¢,* should be positive — that is, &* € R*. 


5.4.2.2.2 Mesh sampling and weighting 

ECSW determines simultaneously the reduced mesh represented by € = {@,,6,..., 
€n,<<n,} C E and its associated set of element weights Goi by training either ap- 
proximation (5.60) or (5.61) — for example, the approximation (5.60) — using a set of 
precomputed reduced, nonlinear, force balance snapshots of the form 


b,(un(t's p); pi) = W' by, (Vun(t's p); p) = W’ by, (Vun) (5.62) 


5 Computational bottlenecks for PROMs: precomputation and hyperreduction —— 217 


fors = 1,...,n,, where t* denotes the k-th sampled time instance and yy, the l-th param- 
eter vector sampled in the parameter space P; the SUSIE s designates a solution 
snapshot that is precomputed at some time instance ge [t°, T] and for some sampled 
parameter point py, € P, and is introduced to simplify the notation; and n, denotes the 
total number of precomputed snapshots. 

For both convenience and computational efficiency, no snapshot u> of the solu- 
tion of the parametric, nonlinear PROM (5.6) is computed in order to evaluate (5.62). In- 
stead, snapshots uy, of the solution of the parametric, nonlinear FOM (5.1) are first col- 
lected as in any standard PMOR method (this approach for achieving computational 
efficiency is identical to that adopted in the case of DEIM, where the snapshots (5.45) 
are used instead of their counterparts (5.44)). Next, these high-dimensional solution 
snapshots are converted on-the-fly into the form Vu}, using an orthogonal projection 
operator onto the subspace spanned by the columns of V. Such a projection operator 
is denoted here by II. 

Let c e(Uy, ) = Lb, (Lewy, ) denote the contribution of the mesh entity e to 
b N, (Uy, ). Given a collection of n, high-dimensional solution snapshots S = {uy, ys 
consider 


s=1? 


(Ty uy, ) = $ wcl Tyuy,)> S= 1... 
ecE 


The above expression can be written in matrix form as 


C1= d, 
where 

W'e,(yuy,) -- Wc, (Myuy,) 

C= a, ; e REO, (5.63) 
weg uy, you. Wc, yuy) 
b, (yay, ) 

d= e R™, (5.64) 
b, (lyuy, ) 


and 1 is the vector whose entries are all equal to 1. It follows that the hyperreduced 
ECSW approximation b, ~ b, of the form given in (5.60) can be written in matrix form 
as 


Ccé* = d, (5.65) 


where £* «€ IR” denotes the vector of element weights extended to the entire mesh 
represented by £. Therefore, this vector contains 0 in each of its rows associated with 
a mesh entity e € E\E. 
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For large-scale high-dimensional models, the practical number of precomputed 
reduced, nonlinear, force balance snapshots n, and the desired dimension of the 
PMOR n are such that nn, < ne. Hence, for all practical purposes, the linear system of 
equations (5.65) can be considered to be often underdetermined. 

The result (5.65) suggests that the pair of minimal subset of sampled elements £ 
representing the desired reduced mesh and associated vector of element weights £* 
that delivers sufficiently accurate hyperreduced approximations of the forms given in 
(5.60) and (5.61) is given by the solution of the following optimization problem: 


* = argmin |ğlo s.t. IC -dl <td, if W’Ky, Vis SPD, 
$20 (5.66) 


f* =argmin |ğllo s.t. IIC -dll <rTtldl, otherwise, 


where t € R* is a small, relative tolerance that can be used to control the accuracy of 
the resulting hyperreduction. Unfortunately, both optimization problems described in 
(5.66) are NP-hard. Therefore, the solution of either of these two problems is infeasible 
for practical meshes. 
Alternatively, inexact solutions to three different convex approximations of (5.66) 
that promote sparsity in the solution can be considered [14]: 
1. Approximation A1 below, which transforms problem (5.66) into a (nonnegative) 
least-squares (NNLS) problem 
l č* x arg min 1C -dl if W’Ky, V is SPD, ‘ach 
é* = arg min ZIC -dij otherwise. 


2. Approximation A2, which is based on the l;-norm and transforms the original op- 
timization problem into a (nonnegative) variant of the basis pursuit problem [16] 
é* ~argmin|é|, s.t. Cě=d ifW’Ky Y is SPD, 
l g0 7) ‘i (5.68) 
é* =argmin|é|, s.t. C&=d otherwise. 


3. Approximation A3 below, which corresponds to transforming problem (5.66) into 
a (nonnegative) regularized, least-squares problem 
Ps in |C - dl +A if W’ Ky, V is SPD, 
l f° =~arg ma IC -dl +All i m, V is ees) 
é* = argmin ||C& — dj} + Allı otherwise, 


where À is a positive penalty parameter. 


In each of these three approximations, the inexactness of the solution specifically 
refers to the fact that the optimal solution is not required to satisfy exactly the Karush- 
Kuhn-Tucker conditions, but is considered to be acceptable if it satisfies instead the 
following conditions: 


C&é-dl,<tlld|, and #20 if W’KyVisSPD, 
| |C§ - dll < Tlidll2 g N (570) 


|C& - dll < Tlidll2 otherwise. 
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Each of the optimization problems (5.67), (5.68), and (5.69) can be solved using 
a parallel, iterative, active set algorithm equipped with (5.70) as a stopping criterion. 
Detailed descriptions of such algorithms can be found in [14], where it was shown that 
the NNLS approach (5.67) (and its active set algorithm) is by far the fastest and most 
computationally efficient approach for performing mesh sampling and weighting. 

For completeness, Algorithm 5.4 summarizes the computation of the reduced 


Cre 


mesh represented by £ and its associated set of element weights {&,* loos 
wl 


Algorithm 5.4: ECSW: offline and online phases. 
Offline phase 
Input: S, T 
Output: £, ee" hos, 
1: Assemble C from S using (5.63) 
2: d- C1 
3: Solve (5.67), (5.68), or (5.69) for §* 
4 Ee indices(§ *> 0) 
oa as a EFO 
Online phase 7 
Input: u`, u,(t; H“), E, eS 
Output: b, (u, (t; u*); p*), K (t; p“); p“) 
1: Compute b,,(u,,(t; uw); p*) using (5.60) 
2: Compute K,,(u,,(t; u*); p*) using (5.61) 


Cre 


Remark 5.5. If the semi-discretization is of the FV or FD type and, Ve € £, the evalua- 
tion of the quantities b,(L,, Vu,(é 4); 4) and K,(L,, Vu, (t; 4); p) requires the local- 
ization matrices L,, Ve € €, the construction of the reduced mesh must be upgraded 
as follows. After € has been computed as described above, it must be augmented with 
the entities that define for each e € £ the Boolean matrix L,+ so that all hyperre- 
duced computations can be performed completely on the upgraded reduced mesh. 
This process is similar to that described in Section 5.4.2.1.5, where the sample index 
set Z is augmented with the entities required to construct the sampled quantities at 
these indices in order to form the final reduced mesh. This additional step is not re- 
quired in the case of an FE semi-discretization, because in this case, the evaluations 
of b,(L,Vu,(t; 4); p) and K,(L, Vu,,(t; 4); p) are local to the element e. 
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5.4.2.2.3 Structure-preserving property and significance 


Let 
Ny, =Np/2, 
åy, (t; S 
t; = h 
ey fae p) 
fr ; fy , 
fy, (ty, (ts Ws H aces (ay, (t; Ws W) + fy (Gy, (6; Ds g 6.71) 
Gy, (t; w 
i a Hear H) 
Sn, (6; H) ; 
M (0) 
sede ny OO a 
where: 


qy, (ts W9) denotes the parametric, displacement/rotation vector associated with 
a parametric FOM resulting from the FE semi-discretization of a nonlinear, non- 
conservative, second-order dynamical system such as, for example, a nonlinear, 
nonconservative, structural dynamics system, and dy, (É; H) denotes the corre- 
sponding velocity vector. 

f(y, i(t; y); p) denotes the parametric, nonlinear, true internal force vector as- 
sociated with the aforementioned FE-based FOM, and the word “true” is used here 
and throughout the remainder of this section to distinguish a newly defined en- 
tity from its generic counterpart introduced at the beginning of this chapter. This 
force vector usually derives from a parametric, nonlinear, internal potential en- 
ergy vay, (t; W; p) - that is, 


int T 


fy (Ay, (f; W; H) =- (an, (t; p); p). (5.72) 


fy (dy, :(t; p); p) denotes the parametric, nonlinear, dissipative force vector as- 


ae with the aforementioned FE-based FOM. Typically, this vector contains 
dissipative forces that remain parallel and in opposite direction to the velocity 
vector, depend on its modulus, and are associated with a parametric, nonlinear, 
dissipation function Dàn,’ (t; p); p) that is a homogeneous function of order min 
the velocity vector — that is, 


dy, (t; Ms (änt H); H) = mD(Gy, (t; p): p). 


For example, m = 1 corresponds to the case of dry friction, m = 2 to viscous damp- 
ing, and m = 3 to aerodynamic drag. Hence, the parametric, nonlinear, dissipative 
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force vector can be written as 
Desc 
=—(qw, (6 H): H). (5.73) 


o 
fr åd (ts ); 
(¢ N, 5 H u) = Ody, 


fc ext th H) denotes the parametric, conservative force vector deriving from a para- 


ionic external potential V ayy (t; y); p) — that is, 


ext T 


f= (f; p) =- (any (t; W; B) © V™ (dy, 6 W: H) = -ayfer (6 W. 
Nh Oqn' h 
(5.74) 


p _(t; u) denotes the parametric, nonconservative force vector. 
Nh 


- My, denotes the parametric, true mass matrix of dimension N,’ associated 
with the aforementioned FOM. 


In this context, the semi-discrete, parametric FOM (5.1) models a nonlinear, noncon- 
servative, second-order dynamical system — for example, a nonlinear, nonconserva- 
tive, structural dynamics system — whose governing FE equation 


My: (4) 4y,' +f r (Gy, (t; PD; B) + fy, (Ay, (6 wW; p) = fen, (t; H) + fas, (tp) (5.75) 


has been rewritten in first-order form. Note that the emphasis here on the case of a 
nonconservative system is simply because it is more general than the particular case 
of a conservative system. 

Let 


V(ay, (6 Ws H) = Y™ (ay, (t; ws p) + V (dy, (t ws p) (5.76) 
denote the parametric, nonlinear, total potential associated with the second-order dy- 
namical system represented by (5.75), and let 

f 1. . 
T (y,'(6 p); p) = Sån, (6 H)My, (Wy, (6 W (5.77) 


denote its kinetic energy. For this second-order dynamical system, Hamilton’s princi- 
ple can be written as 


T(y,1(6 W); p) + V(dy, (t; W; p) = -mD (dy, (t; W); p) + am fne, (6 p), (5.78) 


where 7 is the kinetic energy defined in (5.77), V is the total potential defined in (5.76), 
(5.72), and (5.74), and D is the dissipation function defined in (5.73). The associated 
Lagrange equation of motion can be written as 
d (2 oT 
dt Ody, 


T 


ð 
- )(äw 6 ws 9) - 2 (ayy p); p) - Z yt p); p) 
ay, 


aD 
Ody, 
ee H) = (5.79) 
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In the special context defined by (5.71), the above FE-based equation is equivalent to its 
counterpart given in (5.75). More importantly, equations (5.78) and (5.79) collectively 
define a mathematical structure that is referred to here as the Lagrangian structure 
associated with Hamilton’s principle. In [24], it was shown that ECSW preserves this 
structure as in the context of (5.71), the kinetic energy 7 based on the subspace approx- 
imation (5.17) (adjusted to the dimension n’), and the hyperreduced total potential 7 
and dissipative function D obtained by the application of the ECSW approximations 
of the form given in (5.60) to 7 and D, respectively, satisfy 


T (Gv (6 W; p) + V(r (6 W; p) = -mD (Gy (ts Ws p) + Af, 6D (5.80) 


which is similar to (5.78). In (5.80), n’ = n/2, qw is the vector of reduced (or generalized) 
coordinates associated with qy, and fi, (t; p) is the reduced vector of nonconserva- 
tive external forces. 

Hence, for parametric, nonlinear, second-order dynamical systems, ECSW is a 
structure-preserving hyperreduction method. A major consequence is that in the con- 
text defined by (5.71), a preferred time integrator applied to the time discretization of 
the parametric, nonlinear PROM (5.6) hyperreduced by ECSW will exhibit the same, if 
not better, numerical stability properties as those that it exhibits when applied to the 
time discretization of the underlying FOM (5.1) — see [24] for the justification. Specifi- 
cally, if the preferred time integrator is energy-conserving and unconditionally stable 
when applied to the time discretization of (5.1), it is guaranteed to be uncondition- 
ally stable when applied to the hyperreduction by ECSW of the parametric, nonlinear 
PROM (5.6). The latter result follows directly from the preservation by ECSW of the 
Lagrangian structure associated with Hamilton’s principle. 


5.5 Applications 


Here, the DEIM and ECSW are illustrated with both academic and real-world, para- 
metric and nonparametric, linear and nonlinear applications for which hyperreduc- 
tion is necessary for achieving computational efficiency. As stated before, these two 
methods represent the state of the art of approximate-then-project and project-then- 
approximate hyperreduction methods, respectively. Throughout this section, a hyper- 
reduced PROM is referred to as an HPROM, and all reported computations are per- 
formed in double precision arithmetic. 


5.5.1 Hyperreduction of a parametric Helmholtz-elasticity model 


Here, the two-dimensional, parametric, acoustic scattering problem graphically de- 
picted in Figure 5.3 is considered. This problem is characterized by: a two-dimensional 
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Figure 5.3: Parametric, one-way coupled, acoustic scattering problem: Variable shape scatterer, 
artificial boundary, computational domain and its discretization, and amplitude of a scattered wave. 


obstacle B parameterized by the shape of its boundary ðB; a computational domain 
Q delimited by the parametric boundary ðB and a nonparametric artificial boundary 
Text = ƏQ\ƏB, where a local absorbing boundary condition is applied; and a pla- 
nar incident wave parameterized by a variable direction (cosa sina) and a vari- 
able wave number x. It is modeled by an FE-based FOM that is a linear version of 
the second-order FOM 5 75) where the solution dy, (6 p) is sought after in the form 
dy, (t; 4) = ap we t, where the superscript amp designates the amplitude of a 


vector, I dinies the pure imaginary number (I? = -1), w = xc is a specified cir- 
cular frequency, and c is the speed of sound in the medium surrounding the obsta- 
cle B; fy (Any (6 9) = Kyy Way, (t WD = Kyy Wag we ET Ony (6 p): p) = 
Sn, Wåny (6 W) = -IwSy, (Wap (Me™ ;and facs, (t; p) = fees. Hence, the 
aforementioned acoustic scattering problem is represented here by a linear FOM of 
the Helmholtz type 


(Ky, 9) - w° My: (H) - IwSy , (9)q ap’ W= fy W, (5.81) 


where Sy, (4) is a real-valued sparse matrix associated with the discretization of the 
local absorbing boundary condition and therefore has nonzero entries on Tįxt only; 
and fr (u) arises from the treatment of the displacement (Dirichlet) boundary condi- 
tion on ðB associated with the incident wave. This FOM is representative of simplified 
versions of Helmholtz problems that arise in many applications pertaining to sonar 
and radar design, medical imaging, and nondestructive testing. 

Specifically, the shape of 0B is parameterized as follows: 


OB(g) = OB + Tar $2» $3» $4)» 
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where B is a disk of center (0,0) and radius 1m — and therefore dB is a circle of center 
(0, 0) and radius 1m -Tz is the shape deformation function defined by the following 
parameterized two-dimensional displacement vector [18]: 


& cos 2s + & cos As — (¢; + ¢3) 
ur, (6) = ( kien | 
Q sins + ¢, sin 3s 
where s = tan '(%) € [0, 27], x and y denote the coordinates of a generic point on the 
circle 0B, andG=(% Q & Q) €PycR*. 

In all cases, the artificial boundary Text is chosen to be the circle of center (0, 0) 
and radius 5 m; hence, Vf € Py, 0Q\0B = 0Q\aB = Text where Q denotes the refer- 
ence computational domain defined by the annular disk of internal radius equal to 
1m and external radius equal to 5m. This reference domain is discretized by a ref- 
erence mesh (0) with ne = 142,168 linear triangular elements. For each nonzero 
value of the deformational vector ur, (¢), this mesh is deformed using the structural 
analogy method described in [37] in order to obtain the mesh (Ç, n) that conforms 
to the parametric boundary AR )and Honperemenic artificial boundary Text. To this 
end, two displacement DOFs uy and uj in the x- and y-directions, respectively, are at- 
tached to each node of .% (0), in addition to the DOF qj governed by the FOM (5.81). 
For each queried parameter point ¢ € Pz, the following high-dimensional problem is 
constructed and solved: 


Ky 0G, muy, ($1) = fy e(n), (5.82) 


where Kyo ,n) is an FE stiffness matrix of dimension N}? associated with the 
elasticity-based structural analogy method described in [37], 7 is a user-defined nu- 
merical parameter of this method, wy oS n) is the vector of displacement DOFs of 
(0), and fy,0 (Ç, n) arises from the treatment of the displacement (Dirichlet) bound- 
ary condition on ðB associated with the deformation of this boundary. Then, ./(¢, n) 
is constructed by updating the position of the nodes of .@(0) using the computed vec- 
tor of displacement DOFs wy 2G ,1). This approach ensures that Vf € Py, the mesh 
topology of the reference mesh ./(O) is preserved by .#(¢, n). Therefore, it simplifies 
the PMOR of the FOM (5.82) whose operators depend on 4 (¢, n). For ne = 142,168, 
N,° = 141,256 and N,’ = 71,324. 

Let Pem C R? denote the five-dimensional parameter space where a typi- 
cal point Mr, is given by Men = G Q & %& n), and let P c R’ denote 
the seven-dimensional parameter space where a typical point u is given by w = 
G 6 & & nN a x)", where it is recalled that a and x parameterize the planar 
incident wave direction and wave number. The considered ranges for the parameter 
space P are: 

- & € [-1/2,1/2], G € [-0.8,12], G € [-0.05,0.05], %, € [-0.05, 0.05], and « 

[0, 1.4]; 
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- ae [0,7/6] and x € [2,4]. 


Then, for each queried parameter point u € P, the solution of the acoustic scattering 

problem graphically depicted in Figure 5.3 consists in solving the one-way coupled 

problem defined by the FOM (5.82) and the FOM (5.81), as follows: 

— First, solve the problem (5.82) for Hen C H and transform ./#(0) into -4(¢,n) 
using the computed Wy, 9 (Hey) 

- Next, construct the problem (5.81) for the queried parameter point u and solve it 
to obtain q Ne (p). 


Because the parametric stiffness matrix Ky, 06 n) of the parametric FOM (5.82) de- 
pends implicitly on Hen 7 and therefore, Kyo ,4) cannot be explicitly written as 
an affine function of p¢,,, — and because the parametric operator Ky, (H) -w° M Ni (H)- 
TwSy, (H) governing the parametric FOM (5.81) depends on the updated mesh .“(¢, n) 
and therefore depends implicitly on Hen CM the hyperreduction of any PROMs con- 
structed for the FOM (5.82) and/or the FOM (5.81) is necessary in order to achieve com- 
putational efficiency. 

The PMOR of the one-way coupled, acoustic scattering problem defined above is 
performed in two steps as follows. First, a PROM for the mesh motion problem (5.82) is 
constructed, followed by the construction of a PROM for the Helmholtz problem (5.81). 
In both cases, the POD and Galerkin projection methods are used for this purpose. 
Specifically, the LATIN hypercube sampling method is applied to sample Pign) < R 
in 50 points, and 50 solution snapshots are computed at these points and compressed 
using SVD. This leads to a PROM for the mesh motion problem of dimension n, = 10. 
Then, the DEIM is applied to hyperreduce this PROM using seven POD basis vectors 
for approximating the action of an instance of the left-hand side matrix K N2 (¢,7) on 
a vector of dimension N ie and 21 POD basis vectors for representing a right-hand side 
vector of the form fy, (¢,7). A relative tolerance of 10°° is used to truncate all com- 
puted POD bases [32], which is reasonable given the decay of the normalized singular 
values reported in Figure 5.4 for each computed snapshot matrix. 

Next, P c R’ is similarly sampled at 250 points in order to compute 250 solu- 
tion snapshots of the Helmholtz problem (5.81). To this end, the PROM constructed 
for the mesh motion problem (5.82) is used for solving this problem at each sampled 
point and updating accordingly the position of the nodes of .#@(0). The 250 computed 
solution snapshots of (5.81) are then compressed using SVD to obtain a global ROB 
and associated Galerkin-PROM of dimension n, = 137. Then, this PROM is hyperre- 
duced using DEIM, 206 POD basis vectors for approximating the action of an instance 
of the left-hand side matrix (K N H) - w’ My; (p) —IwSy, (4)) on a vector of dimension 
Nj, and 229 POD basis vectors for ep Sees a right-hand side vector of the form 
f N W). In this case, a relative tolerance of 107 is used to truncate all computed POD 
bases: This tolerance value is reasonable given the decay of the normalized singular 
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Figure 5.4: Parametric, one-way coupled, acoustic scattering problem: Decay of the normalized sin- 
gular values of the matrix of snapshots of the right-hand side vector of (5.82) (left), of the action of 
the left-hand side matrix of (5.82) on a vector (middle), and of solutions of (5.82) (right). 
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Figure 5.5: Parametric, one-way coupled, acoustic scattering problem: Decay of the normalized sin- 
gular values of the matrix of snapshots of the right-hand side vector of (5.81) (left), of the action of 
the left-hand side matrix of (5.81) on a vector (middle), and of solutions of (5.81) (right). 


values reported in Figure 5.5 for each computed snapshot matrix. Note that this decay 
is significantly slower than in the case of the mesh motion problem, which indicates a 
stronger dependence of the operators and solution of the Helmholtz problem (5.81) on 
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Figure 5.6: Parametric, one-way coupled, acoustic scattering problem: DEIM-based reduced mesh. 


their parameters. Figure 5.6 shows the reduced mesh obtained for this one-way cou- 
pled Helmholtz problem using the DEIM. This mesh contains 1,391 elements, which 
corresponds to 0.98 % of the total number of elements n, of the original mesh. The el- 
ements of this reduced mesh are located mainly around the scatterer (obstacle) — that 
is, in the region of the computational domain where the sensitivity of the solution of 
problem (5.81) to shape variations is higher. 

Figures 5.7 and 5.8 display solutions of the mesh motion and Helmholtz problems 
computed using the DEIM-based HPROMs for several sampled points of the parameter 
space P c R’. Figure 5.7 also reports, for 100 parameter points sampled in Pen) © R, 
the convergence of the average global relative error of the HPROM-based solution of 
the mesh motion problem — which is defined here with respect to the FOM-based coun- 
terpart solution, measured in the energy norm over the entire mesh, and averaged over 
the number of sampled parameter points — as a function of the dimension n,, of the 
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Figure 5.7: Parametric, one-way coupled, acoustic scattering problem: Visualization of the magni- 


tude of the mesh displacement for different parameter values (left); and variation of the relative 
error of the mesh motion HPROM with the dimension n, of the associated PROM (right). 
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Figure 5.8: Parametric, one-way coupled, acoustic scattering problem: Visualization of the scat- 
tered wave amplitude for different parameter values (left); and variation of the relative error of the 
Helmholtz HPROM with the dimension ng of the associated PROM (right). 


PROM underlying the HPROM. Similarly, Figure 5.8 shows the convergence with n, 
of the counterpart error associated with the solution of the Helmholtz problem. The 
reader can observe that in the case of the Helmholtz problem, the convergence of the 
aforementioned relative error is significantly slower than that of its counterpart for 
the mesh motion problem and eventually flattens at roughly n, = 80. This behavior 
is characteristic of the HPROM error which tends to be dominated by the error intro- 
duced by hyperreduction, and which cannot always be decreased by increasing the 
dimension of the underlying PROM. 

Regarding wall-clock performance, the online solution of this one-way coupled, 
acoustic scattering problem defined at a queried but unsampled parameter point u € 
P c R’ can be computed in 0.43s using the mesh motion and Helmholtz HPROMs on 
a workstation with a dual-core Intel Core i5 processor running at 2.8 GHz and 16 GB of 
memory. On the same computing platform, the online solution of the same one-way 
coupled problem using the FOMs (5.82) and (5.81) is 24 times slower. 


5.5.2 Hyperreduction of a parametric PDE-ODE wildfire model 


Next, a time-dependent, nonlinear, two-way coupled, PDE-ordinary differential 
equation (ODE) system describing the evolution of a wildfire in a domain representing 
the two-dimensional layer just above the ground surface is considered. This system 
models a wildfire using balance equations for energy and fuel [39, 30]. In principle, 
the PDE is the two-dimensional unsteady, advection-diffusion equation governing 
temperature. However, for demonstrative purposes the advection term is neglected 
here and therefore the PDE is the two-dimensional unsteady diffusion equation gov- 
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erning the temperature distribution. The ODE governs the time-dependent fuel supply 
mass fraction. Two-way coupling between the two equations is performed via non- 
linear source terms (see [39, 30] for further details). For this problem, the parameters 
of interest are: the thermal diffusivity, w, in m?/s; the rate of temperature rise at the 
maximum burning rate, A, in K/s; the proportionality coefficient in the modified Ar- 
rhenius law, B, in K; the scaled coefficient of heat transfer to the environment, C, in 
K"'; and the relative fuel consumption rate, Cs, in s™t. They define a parameter space 
P c R? and are stored in a parameter vector u € P. The parametric solutions of this 
problem propagate in three areas of localized combustion: a preheated area ahead 
of the fire; a combustion zone; and a burning region behind the fire. Their efficient 
computation using PROMs requires hyperreduction, because the coupling source 
terms are nonlinear and their dependencies on the aforementioned parameters are 
nonpolynomial. 

The computational domain Q c R° is chosen to be a square of side 1,000 m. The 
ranges of the individual dimensions of P are set as follows: 0.15 < w < 1.5;2< A < 20; 
60 < B < 600; 4.5 x 10° < C < 45x10; and 1.5 x 107 < Cs < 1.5 x 101. Note 
that for increasing values of A/Cs, the temperature in the traveling combustion wave 
increases; for increasing values of y, both the width and the speed of the combustion 
wave increase; and a sustained combustion requires sufficiently small values of Cs. 

In order to model a fire located in a small circular region close to the center of Q, 
the temperature in this chosen computational domain is initialized as 


T(x, 0) = T; eae To XEQ, 


where T, = 1,200 K, T, = 300 K, and x = 10‘ m. In order to model the fuel depletion 
of a fully developed fire, the fuel supply mass fraction is initialized as 


S(x, 0) = 1— eS, 


where Xe = 5x10? m. Fort > 0, the reaction heat spreads isotropically, heating the fuel 
ahead of the wave until the reaction in front of the wave can sustain itself, thus causing 
the spread of combustion. At the rear, the reaction ceases due to fuel depletion, thus 
causing the temperature to decrease due to cooling. 

A two-dimensional FE model of this coupled PDE-ODE system is constructed us- 
ing a triangulation of Q with n, = 137,820 elements and n, = 69,313 nodes. At each 
node, two DOFs are attached: one for the temperature, and one for the fuel supply 
mass fraction, which results in a total number of N = 138,626 DOFs. Spatial approx- 
imation is performed using piecewise linear finite elements. Temporal discretization 
is performed using the first-order time-accurate backward Euler method, and all non- 
linear terms are treated semi-implicitly. All numerical simulations reported below are 
conducted in the time interval [0, 750] s using the fixed time step At = 5s. 

PMOR is achieved using the POD-based Galerkin projection method. For this pur- 
pose, solution snapshots are computed using the parametric FOM described above 
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at each time step, for 100 randomly sampled points in P. Next, these snapshots are 
compressed to construct an ROB and then an associated PROM of dimension n = 12. 
The DEIM is applied to hyperreduce the PROM by approximating only the nonlinear 
terms, as all linear terms of the FOM depend affinely on the problem parameters and 
therefore do not require hyperreduction. Specifically, a basis of dimension m = 11is 
constructed using snapshots of the nonlinear (coupling) source terms computed at 
each time step, for 50 randomly sampled points in P, and a set of 11 interpolation 
indices Z (|Z| = m = 11) is selected. The corresponding reduced mesh £ has only 45 el- 
ements — that is, 0.033 % of the number of elements in the original FE mesh. Using the 
resulting HPROM, the pointwise approximation of the ODE governing the fuel supply 
mass fraction is advanced only at the DOFs corresponding to the interpolation indices 
in Z, which further decreases the processing time for this application. 

Figures 5.9 and 5.10 contrast the FOM-, PROM-, and HPROM-based predictions of 
the temperature time histories and fuel supply mass fraction at four different spatial 
locations, for two significantly different parameter configurations. They show that in 
each case, the FOM, PROM, and HPROM deliver essentially the same results, which in- 
dicates that both PMOR- and hyperreduction-induced errors are minimal. These two 
figures also reveal that the two different parameter configurations lead to very dif- 
ferent physical results, which, given also the previous remark, illustrates the robust- 
ness of the global PROM and that of the associated HPROM with respect to parameter 
changes. Figure 5.11 reports for both parameter configurations the temperature distri- 
butions predicted at t = 60s using the HPROM and the corresponding absolute errors 
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Figure 5.9: Parametric, two-way coupled, PDE-ODE wildfire problem: Temperature (left) 
and fuel supply mass fraction (right) predicted at four different spatial locations fory = 
(1.07, 91.19, 258.41, 107“, 0.0201)”. 
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Figure 5.10: Parametric, two-way coupled, PDE-ODE wildfire problem: Temperature (left) and fuel 
supply mass fraction (right) predicted at four different spatial locations for u = (1.5, 50, 65,5.5 x 
107, 0.15)". 


(measured with respect to the FOM-based temperature solutions). It leads to the same 
conclusions as Figures 5.9 and 5.10. 

Table 5.2 reports the wall-clock timings obtained on a workstation with a dual-core 
Intel Core i5 processor running at 2.8 GHz and 16 GB of memory, the speedup factors, 
and the average global relative errors of the PROM- and HPROM-based simulations. 
These errors are defined here with respect to the results of the FOM-based counter- 
part simulations, measured in the energy norm over the entire mesh, and averaged 
over 50 different parameter points randomly selected in P. The reported results show 
that overall, the DEIM-based HPROM maintains the level of accuracy of its underlying 
PROM. As expected, the PROM does not accelerate the FOM-based simulation by any 
meaningful factor, but the HPROM delivers a speedup factor of almost 30. 


Table 5.2: Parametric, two-way coupled, PDE-ODE wildfire problem: Wall-clock timings on a single 
core and speedup factors. 


Wall-clock time (s) Speedup factor Relative error 
FOM 88.4 1.0 -= 
PROM 48.5 1.82 3.16x107 
HPROM 3.18 27.8 


546x107 
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Figure 5.11: Parametric, two-way coupled, PDE-ODE wildfire problem: Temperature distributions 
att = 60s predicted using the HPROM (left) and corresponding relative errors (right), fory = 
(1.07, 91.19, 258.41, 10-4, 0.0201)! (top) and u = (1.5,50, 65, 5.5 x 1077, 0.15)” (bottom). 


5.5.3 Hyperreduction of nonlinear structural dynamics models 


Two highly nonlinear structural dynamics problems are considered here. The first one 
focuses on a fast spinning top: It has the merit of being easily reproducible by the 
reader who is familiar with solid mechanics, and is explored to highlight the instabil- 
ity of the DEIM for second-order dynamical systems. The second problem focuses on 
an underbody blast event. Its associated FE model is representative of two families of 
computational models: those whose practical exploitation calls for model reduction, 
due to their CPU-intensive nature; and those that are difficult to reduce due to the pres- 
ence of rotational DOFs. In both problems, the PROMs are constructed using the POD 
method based on displacement/rotation snapshots. In the first problem, the PROM is 
hyperreduced using both DEIM and ECSW. In the second problem, only ECSW is ap- 
plied to hyperreduce the constructed PROM, due to its superior numerical stability 
properties. Because DEIM and ECSW operate essentially in the same fashion whether 
the PROM to be hyperreduced is parametric or nonparametric, both aforementioned 
problems are considered here in their simpler, nonparametric context (for the para- 
metric case, the reader is referred to [31]). In all cases, ECSW is configured with the 
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convex approximation A1 (5.67). Furthermore, it is equipped with the NNLS algorithm 
developed by Lawson and Hanson [28] for solving the NNLS problem associated with 
this approximation because this algorithm has demonstrated an excellent track record 
of robustness and performance for hyperreduction [22, 24, 14]. 

Given a computed, time-dependent, PROM-based approximate solution GRA (t) = 
V'q,)(t), the corresponding global relative error in a direction © is defined here as 


Erra, (O) - V'a, (O)" do, (®© - Vao, ©) 
: : x 100%, 


RE, = 
Erer doy (07a, (0 


where the subscript © designates the displacement/rotation in the x-, y-, or z-direction 
of the global coordinate frame; do, (t) is the vector of ©-displacement/rotation DOFs 


at time t extracted from the solution obtained using the discrete FOM of interest; 
Vv! do: (t) is the vector of }-displacement/rotation DOFs at time t extracted from the 
solution obtained using the PROM or hyperreduced PROM whose performance is be- 
ing assessed; and P is the set of timestamps used in the evaluation of RE, — that 
is, 


P = ft € {t°,t° + As, t? + 2As,...}:t < T}, 


where As denotes the sampling time interval chosen for the evaluation of the global 
relative error. 

All computations reported herein are performed in double-precision arithmetic 
on a parallel Linux cluster where each computing node consists of two quad-core Intel 
Xeon E5345 processors running at 2.33 GHz inside a Dell Poweredge 1950 and has 16 GB 
of memory, and the interconnect is Cisco DDR InfiniBand. 


5.5.3.1 Fast spinning top in a gravitational field 


First, the hyperreduction of a geometrically nonlinear, FE-based PROM constructed 
for a fast spinning top in the presence of a gravitational field is considered. This hy- 
perreduction problem was previously discussed in [22], [24], and [14]. The shape of 
the top is a cone with height H = 0.1m and radius R = 0.05m. The top is assumed 
to be made of aluminum, which is modeled here using the Saint Venant-Kirchhoff 
constitutive law. This hyperelastic material law can be expressed as a linear relation 
between the second Piola—Kirchhoff stress tensor and the Green—Lagrange strain ten- 
sor. Hence, the internal force vector is in this case a nonlinear function fy" (dy, (t)) 
of the displacement vector q m, (© due to the nonlinear kinematics (geometric nonlin- 
earities). The Young modulus of this material is E = 80 GPa, and its Poisson ratio is 
v = 0.33. Its density in the reference configuration is p, = 2, 700 kg/m’. 
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The top is simply supported at its apex. Initially, it is set into the position obtained 
by a rigid body rotation of 3 rad about the x-axis, and into the spinning motion about 
its axis with the convected angular velocity Q, = 300 rad/s. The gravitational accel- 
eration, g = 9.81m/s’, generates an external body force that acts on the top in the 
negative z-direction. 


Figure 5.12: Spinning top: FE mesh and nodes 99 and 608 where 
the time histories of the x- and z-displacements are observed, 
respectively. 


For this problem, an FE structural model is constructed using 4,461 10-noded tetrahe- 
dral elements with three displacement DOFs per node — which results in an FOM of 
dimension N, = 13,317 DOFs. Figure 5.12 shows the constructed FE model. Time dis- 
cretization is performed using the second-order accurate explicit central difference 
method. For this explicit time integrator and the aforementioned FOM, the maximum 
stability time step is At = 1x 10~’ s. Using this time step, the FOM-based simulation of 
the first 1 second of top spinning consumes 21.8 hours of wall-clock time on 80 cores 
of the Linux cluster. The time-dependent solution computed during this simulation is 
sampled every As = 2x10 s for the purpose of constructing three different POD-based 
PROMs of dimension n = 9, n = 10, and n = 15. The DEIM and ECSW are applied for 
the hyperreduction of each of these PROMs as follows: ECSW is applied using values 
of the relative training tolerance t that produce reduced meshes containing less than 
1% of the number of elements of the original FE mesh; and DEIM is applied with the 
set of indices Z selected so that if one DOF is sampled at a node, then all DOFs attached 
to that node are also sampled — and therefore, |Z| > m, where m is the dimension of 
the ROB U (5.39). 

All HPROM-based simulations of the dynamics of the spinning top are performed 
using the same explicit central difference time integrator used for computing the FOM- 
based time-dependent solution. However, because these simulations are not governed 
by the same Courant—Friedrichs—Lewy restriction as their counterpart FOM-based 
simulation, they can be performed — and therefore are performed — using a time step 
that is 20 times larger than the maximum-stability time step of that reference simula- 
tion. Furthermore, because all reduced meshes generated by DEIM and ECSW contain 
in this case less than 35 elements (for example, see Table 5.4 for the case of ECSW), 


5 Computational bottlenecks for PROMs: precomputation and hyperreduction —— 235 


all HPROM-based simulations of the dynamics of the spinning top are executed ona 
single core of the Linux cluster. 

It turns out that for this problem and chosen time integrator, all discrete HPROMs 
obtained using the DEIM are numerically unstable, independently of the chosen value 
for m; for example, see Figure 5.13, which reports the computed time histories of the 
x-displacement at node 99 of the FE mesh. On the other hand, all counterpart discrete 
HPROMs delivered by ECSW are found to be numerically stable, as anticipated by the 
theory presented in Section 5.4.2.2.3. Figure 5.14 — which reports the computed time 
histories of the x-displacement at node 99 and z-displacement at node 608 of the FE 
mesh — and Table 5.3 show that all discrete HPROMs constructed using ECSW deliver 
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Figure 5.13: Fast spinning top: Numerical instabilities exhibited by the DEIM-based HPROMs. 


Table 5.3: Fast spinning top: Global accuracy delivered by the ECSW-based HPROMs. 


n RE, (%) RE, (%) RE, (%) RE ye, (%) RE ye, (%) RE ye, (%) 


9 0.012 0.0099 0.014 0.28 0.31 0.17 
10 0.010 0.0083 0.012 0.24 0.26 0.16 
15 0.0078 0.0064 0.0091 0.12 0.13 0.092 
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Figure 5.14: Fast spinning top: Numerical stability and local accuracy delivered by the ECSW-based 
HPROMs. 


Table 5.4: Fast spinning top: Variations with the dimension n of the right ROB of the size of the re- 
duced mesh generated by ECSW and the speedup factor delivered by the ECSW-based HPROM. 


n Reducedmesh Speedup factor 
(# of elements) 


9 12 9.25 x 10° 
10 15 7.66 x 10° 


15 34 3.21 x 10° 


excellent levels of local and global accuracy, respectively. Table 5.4 shows that they 
also deliver impressive speedups: partly because the underlying reduced meshes have 
very small sizes; and partly because these ECSW-based, explicit, discrete HPROMs af- 
ford a stability time step that is on average 20 times larger than the critical time step 
for the underlying discrete FOM. 


5.5.3.2 Structural response of a vehicle frame to underbody blast 


Next, the hyperreduction using ECSW of a highly nonlinear, POD-based PROM of di- 
mension n = 100, constructed for the analysis of the structural response of a generic 
V-hull vehicle frame to a live fire blast test, is considered. The frame has a complex 
structure (Figure 5.15(a)) that is made predominantly of steel, which is modeled here 
as a nonlinear elastoplastic material. It is subjected to a gravity load, and to an ex- 
ternal, configuration- and time-dependent pressure force due to the explosion of a 
10 kg charge placed under its body, at the location shown in Figure 5.15(b). For this 
purpose, a structural, nonlinear, FE model of the vehicle frame is constructed using 
Nne = 236,995 flexible shell and rigid beam elements with six DOFs per node (Fig- 


5 Computational bottlenecks for PROMs: precomputation and hyperreduction —— 237 


(b) charge location 


(d) bai: E 


Figure 5.15: Underbody blast of a V-hull vehicle frame: (a) complex structure; (b) FE mesh, and loca- 
tions of charge and a nodal probe; (c) snapshot of the structural response att = 0.002 s; and (d) 
ECSW-generated reduced mesh. 


ure 5.15(b)). This FOM features large rotations, large strains, large angular velocities — 
and therefore both geometrical and material nonlinearities — and a large dimension 
equal to N, = 1,399,056 DOFs. The blast pressure loading is modeled using the CON- 
WEP software [27]. 

The vehicle is assumed to be initially at rest. For the sake of diversification, time 
discretization is performed here using the implicit generalized-a method [17] with B = 
0.444, y = 0.833, a = 0.333, a, = 0, and the constant time step At = 1 x 10~°s. As for 
the previous problem, ECSW is configured with the convex approximation A1 (5.67), 
equipped with the NNLS algorithm [28] for solving the NNLS problem associated with 
this approximation, and applied to the hyperreduction of the constructed nonlinear 
PROM. Specifically, the relative training tolerance is set to T = 0.01. In this case, the 
NNLS algorithm delivers the reduced mesh £ with 3,145 elements — that is, with about 
1.32 % of the number of elements of the original FE mesh — as shown in Figure 5.15(d). 
Then, the structural response of the vehicle frame is computed three times in the time 
interval [0, 1077] s: using the discrete FOM of dimension N, = 1,399,056, the discrete 
PROM of dimension n = 100, and the ECSW-generated discrete HPROM. All three sim- 
ulations are performed on a single core of the Linux cluster, using the same fixed time 
step At = 1x 10™ s. For the PROM- and HPROM-based predictions, the relative global 
errors are calculated using the same sampling time interval As = 1x10~ s. A snapshot 
of the structural response of the structural system computed using the discrete FOM 
is shown in Figure 5.15(c). 

Tables 5.5 and 5.6 report the wall-clock timings, speedup factors, and global rela- 
tive errors of the PROM- and HPROM-based simulations. These results show that over- 
all, the ECSW-based HPROM maintains the level of accuracy of its underlying PROM. 
This level of accuracy is high for the predicted displacement and rotation fields, rea- 
sonable for the computed velocity field, but low for the predicted angular velocity 
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Table 5.5: Structural response of a vehicle frame to underbody blast: Wall-clock timings on a single 
core and speedup factors. 


Wall-clock time (s) Speedup factor 


FOM 5.02 x 10° 1.0 
PROM 3.63 x 10° 1.38 


HPROM 3.57 x 10° 141 


Table 5.6: Structural response of a vehicle frame to underbody blast: Global relative errors. 


PROM HPROM 
RE, (%) 3.05 3.53 
RE, (%) 2.46 2.72 
RE, (%) 3.52 3.69 
RE sot, (%) 1.53 1.87 
RE rot, (%) 1.63 2.04 
RE jot, (%) 2.40 2.95 
RE vei, (%) 12.05 13.62 
RE el, (%) 7.21 8.38 
RE yet, (%) 4.81 5.70 
RE yel,,,, (%) 23.03 24.89 
RE yel,.,, (%) 24.60 26.63 


RE ye, (%) 29.92 32.40 


rotz 


field. The latter can be improved by including velocity snapshots in the snapshot ma- 
trix underlying the construction of the POD-based PROM. The reader can observe that 
as expected, the discrete PROM does not speed up the solution time of the discrete FOM 
by any meaningful factor. On the other hand, the HPROM reduces the solution time of 
the FOM by more that two orders of magnitude. While significant, this speedup factor 
is smaller than that achievable in the context of explicit discretizations. The reason is 
that like in the case of an implicit discrete FOM, the time step of an implicit discrete 
PROM or HPROM is limited by accuracy and not by stability considerations, and there- 
fore cannot exceed that of the underlying discrete FOM. In other words, the speedup 
factor of 141 achieved in this case is purely due to model reduction and hyperreduc- 
tion. 

Figures 5.16 and 5.17 display the computed time histories of selected displace- 
ment/rotation and velocity/angular velocity DOFs, respectively, computed at a probe 
located at a node of the FE mesh in the vicinity of the explosive charge (Figure 5.15(b)). 
The accuracy levels demonstrated in these two figures for the constructed PROM and 
HPROM are consistent with the quantitative accuracy results summarized in Table 5.6. 
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Figure 5.16: Structural response of a vehicle frame to underbody blast: Sample displacement and 
rotation time histories computed at a probe. 
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Figure 5.17: Structural response of a vehicle frame to underbody blast: Sample velocity and angular 
velocity time histories computed at a probe. 


5.6 Summary and conclusions 


For many linear and nonlinear problems, PMOR is not guaranteed to accelerate the 
performance of N,-dimensional FOMs, even when the resulting PROM has a dimen- 
sionn «x N,. For linear problems, this is the case for large-scale, parametric FOMs, 
where the computational complexity of the projections of the linear operators and 
source term defining the FOM — which must be repeated every time a parameter is 
changed - scales as O (N n'n) and O (N,n), respectively. Therefore, these repeated pro- 
jections can rapidly become overwhelming. For nonlinear problems, this issue is even 
more severe as it arises for both parametric and nonparametric problems. 
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In this chapter, two divide-and-conquer approaches for addressing the computa- 
tional bottlenecks outlined above have been reviewed. 

The first approach is feasible and exact for: parametric, linear FOMs that admit 
an efficient parameter-affine representation, where efficiency in this context has been 
properly defined in this chapter; and parametric and nonparametric, nonlinear FOMs 
characterized by a low-order polynomial dependency of the internal force vector on 
the solution and a time-independent external force vector. This first approach consists 
in dividing the computation of the reduced quantities, whenever possible, into two 
parts: one part that is responsible for the computational bottlenecks mentioned above 
and can be addressed by offline precomputations; and another part that is amenable 
to real-time processing. 

The second approach consists of a family of inexact approaches that introduce 
into the construction of a PROM an additional layer of approximations that enable 
the real-time evaluation of all of its reduced quantities. In this chapter, two method- 
ologies belonging to this family of approaches have been discussed. The first one tar- 
gets parametric, linear PROMs. It samples offline the parameter space of interest at 
a carefully chosen set of parameter points, and constructs at each sampled point an 
accurate linear PROM. Then, at each queried but unsampled parameter point, it con- 
structs in real-time a linear PROM by interpolating on matrix manifolds the opera- 
tors defining the precomputed PROMs. The second inexact methodology discussed 
in this chapter is known as hyperreduction. It is more comprehensive than the first 
one, as it is equally applicable to parametric and nonparametric, linear and nonlin- 
ear PROMs. This methodology comes at least in two flavors: approximate-then-project 
and project-then-approximate methods. At the time of writing this chapter, the DEIM 
represents the state of the art of approximate-then-project methods, and ECSW repre- 
sents that of project-then-approximate methods. For second-order dynamical systems 
such as those arising in wave propagation, solid mechanics, and structural dynamics 
applications, ECSW is to date the only known hyperreduction method with provable 
structure-preserving and unconditional stability properties. As such, it is superior to 
DEIM for this important class of problems. For other dynamical systems, DEIM and 
ECSW typically exhibit comparable performances in terms of accuracy and computa- 
tional efficiency. For such systems, they have been shown in the literature, and are 
shown in this chapter, to be capable of speeding up the execution time of FOMs by 
factors that are problem-dependent, but typically range between one and three or- 
ders of magnitude if not higher. These hyperreduction methods are robust, practical, 
and a must for the reduction of: parametric, linear FOMs that do not admit an efficient 
parameter-affine representation; parametric and nonparametric, nonlinear FOMs that 
are not characterized by a low-order polynomial nonlinearity in the internal force vec- 
tor; and parametric, linear or nonlinear FOMs where the parameter vector may vary 
within a single simulation. 
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6 Localized model reduction for 
parameterized problems 


Abstract: In this contribution we present a survey of concepts in localized model order 
reduction methods for parameterized partial differential equations. The key concept 
of localized model order reduction is to construct local reduced spaces that have only 
support on part of the domain and compute a global approximation by a suitable cou- 
pling of the local spaces. In detail, we show how optimal local approximation spaces 
can be constructed and approximated by random sampling. An overview of possible 
conforming and nonconforming couplings of the local spaces is provided and corre- 
sponding localized a posteriori error estimates are derived. We introduce concepts of 
local basis enrichment, which includes a discussion of adaptivity. Implementational 
aspects of localized model reduction methods are addressed. Finally, we illustrate the 
presented concepts for multiscale, linear elasticity, and fluid-flow problems, provid- 
ing several numerical experiments. 


Keywords: localized model reduction, reduced basis method, randomized training, a 
posteriori error estimation, basis enrichment, online adaptivity, parameterized sys- 
tems, multiscale problems 


MSC 2010: 65Y15, 65N30, 65N55, 65N15, 35J20, 35J25 


6.1 Introduction 


Projection-based model order reduction has become a mature technique for simula- 
tions of large classes of parameterized systems; for an introduction, we refer to the 
text books and survey [14, 50, 97, 15] and to Chapters 1 to 4 of this volume of Model or- 
der reduction. However, especially for large-scale and multiscale problems the “stan- 
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dard” model order reduction approach as described for instance in Chapter 1 of this 
volume of Model order reduction exhibits several limitations: curse of parameter di- 
mensionality in the sense that many parameters require prohibitively large reduced 
spaces, no topological flexibility, and possibly high computational costs and storage 
requirements in the offline stage for instance due to large computational domains. Lo- 
calized model order reduction methods, which combine approaches from model order 
reduction, multiscale methods, and/or domain decomposition techniques, overcome 
or significantly mitigate those limitations. As a further advantage, they allow using re- 
duced spaces of different dimensions in different parts of the computational domain 
and accommodate (local) changes of the geometry and the partial differential equa- 
tion (PDE) in the online stage. The key idea of localized model order reduction is to 
construct local reduced spaces on (unions of) subdomains of the decomposed com- 
putational domain and couple the local reduced spaces across interfaces either in a 
conforming or in a nonconforming manner. In this chapter we investigate localized 
model order reduction for linear coercive elliptic parameterized problems; inf-sup sta- 
ble problems have for instance been considered in [60] and parabolic and nonlinear 
problems will be briefly discussed at the end of this chapter. 

We discuss both conforming and nonconforming localized approximations. 
Prominent examples for a conforming localization for nonparametric PDEs are the 
partition of unity method [11], the generalized finite element method (GFEM) [8, 7, 11, 
10], and component mode synthesis (CMS) [59, 12, 17], [52, 64]. 

A combination of domain decomposition and reduced basis methods has first 
been considered in the reduced basis element method (RBEM) [76, 77, 74], where the 
local reduced basis approximations are coupled by Lagrange multipliers in a noncon- 
forming manner. The reduced basis hybrid method [62] extends the RBEM by addi- 
tionally considering a coarse finite element (FE) discretization on the whole domain 
to account for continuity of normal stresses in the context of Stokes equations. Alter- 
natively, anonconforming coupling can be realized, say, by penalization asin the local 
reduced basis discontinuous Galerkin approach [66], the localized reduced basis mul- 
tiscale (LRBMS) method [5, 90, 87], the discontinuous Galerkin (DG) RBEM [6], or the 
generalized multiscale discontinuous Galerkin method [29]. The static condensation 
reduced basis element (scRBE) method [61, 60, 37, 103] combines intra-element re- 
duced basis approximations similar to the RBEM with coupling techniques from CMS, 
resulting in a conforming approximation. A similar approach is pursued by the ArbiLo- 
Mod [20] that also allows for arbitrary (nonparametric) local changes of the underlying 
equations and/or the geometry. 

In the context of the proper generalized decomposition method (Chapter 3 of this 
volume of Model order reduction) adomain decomposition strategy has been proposed 
in [55], and in [94] hierarchical model reduction [112, 94, 92, 102] has been combined 
with an iterative substructuring method. 

Concerning the generation of local approximation spaces we focus on empirical 
training (see for instance [37, 10, 103]), i. e., local reduced spaces generated from lo- 
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cal solutions of the PDE, and adaptive basis enrichment. In detail, we present local 
approximation spaces that are optimal in the sense of Kolmogorov and can be con- 
structed by solving a local so-called transfer eigenvalue problem on the space of local 
solutions of the PDE. Optimal local approximation spaces for subdomains have first 
been proposed in [10] and for interfaces and parameterized PDEs in [103]. We will also 
show how those optimal approximation spaces can be approximated by random sam- 
pling [21]. A localizable a posteriori error estimator is crucial for an adaptive enrich- 
ment of the local reduced spaces where the reduced approximation is not accurate 
enough. Such an adaptive basis enrichment is one way to approach “optimal” com- 
putational complexity within outer loop applications such as optimal control, inverse 
problems, or Monte Carlo methods. With this respect, we will also present a framework 
for localized residual-based error control [20, 100] as well as localized a posteriori er- 
ror estimation based on flux reconstruction [39, 90]. 

Naturally, the presented methods for localized model reduction share a lot of fea- 
tures with domain decomposition techniques and multiscale methods. We particu- 
larly refer to domain decomposition and preconditioning techniques with multiscale 
coarse spaces such as [1, 45, 42] or the more recent contributions [106, 43, 47]. In 
the context of the FETI-DP iterative substructuring method we refer to [82, 67]. For 
multiscale problems there has been a tremendous development of suitable numeri- 
cal methods in the last two decades, including the multiscale FE method (MsFEM) 
[54, 35, 36, 49], the heterogeneous multiscale method [114, 115, 85, 2], the variational 
multiscale method [56, 58, 72], or the more recent local orthogonal decomposition 
[81, 48]. Model reduction can be used to accelerate the solution of localized problems 
which occur in multiscale methods; see, e. g., [3, 4]. Similar to the methods presented 
in this chapter the generalized MSFEM (GMsFEM) [34, 31, 30] relies on a Galerkin pro- 
jection on local subspaces, but in contrast uses ideas from multiscale methods to con- 
struct the local bases. A connection between multiscale methods and domain decom- 
position has recently been investigated in [69-71]. 

This chapter is organized as follows. In Section 6.2 we introduce the problem set- 
ting and basic notation for localized model order reduction of coercive variational 
problems. Concepts for conforming and nonconforming coupling of approximation 
spaces are presented in Section 6.3. Section 6.4 deals with the preparation of local 
approximation spaces. Particularly, the construction of optimal local approximation 
spaces and their approximation via random sampling is presented and illustrated with 
numerical experiments for linear elasticity. In Section 6.5 we present two abstract 
frameworks for localized a posteriori error estimation and give exemplifications for 
conforming and nonconforming localized model reduction approaches. Localized a 
posteriori error estimators are the key ingredient for basis enrichment strategies and 
online adaptivity, which are presented in Section 6.6. Computational aspects are dis- 
cussed in Section 6.7 and numerical experiments for multiscale problems and fluid 
flow are presented in Section 6.8. We conclude by showing possible extensions to 
parabolic and nonlinear problems in Section 6.9. 
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6.2 Parameterized partial differential equations and 
localization 


Let QO c R, d = 1,2,3, be a large, bounded domain with Lipschitz boundary. Let us 
further introduce a Hilbert space V such that [H (0) c V c [H'(Q))’, z = 1,...,d, 
and denote by V’ the dual space of V. Moreover, we introduce the compact set of ad- 
missible parameters P c R”, p € IN. We consider the following variational problem. 


Definition 6.1 (Parameterized coercive problem in variational form). For any parame- 
ter u € P find u(p) € V, such that 


a(u(H), Vv; H#) = f(v;p) forallve V. (6.1) 


Here, f(;u) € V’ and a(.,-;) : V x V > R denote parametric linear and bilinear 
forms, the latter being continuous and coercive with respect to the norm ||- ||y induced 
by the inner product (.,-)y : V x V — R. That is, there exist constants O < a < a(p) < 
y(#) < y, such that for any 4 € P, 


a(v,ws#) <y(@) Ivy Iwly forall v,we V, 


av, v; H) > a(p) Ivy, for allv e V. 


Let us denote the energy norm of u for a parameter pas llull; = a(u, u; w” 2. Prob- 
lem (6.1) thus admits a unique solution for all u € P owing to the Lax-Milgram theo- 
rem. Examples for (6.1) include elliptic multiscale problems, incompressible fluid flow, 
and linear elasticity, as detailed in the following. We will consider Neumann boundary 
conditions on Ty and Dirichlet boundary conditions on Tp, where Ty, Tp are nonover- 
lapping and Ty UTp = 0Q. To simplify notations, homogenous boundary conditions 
on 0Q will be prescribed in most places. 


Example 6.2 (Parametric elliptic multiscale problems). With V = HXO), the pressure 
equation in the context of two-phase flow in porous media (obtained through Darcy’s 
law) reads as follows: Given a collection of sources and sinks q € 17(Q) anda para- 
metric and possibly highly heterogeneous permeability field x : P > L°(Q)**4, find 
for each u € P the global pressure u(u) € V, such that 


-V-(k(w)Vu(u)) =q ina weak sense in V’. (6.2) 


If the smallest eigenvalue of x(u) is bounded from below away from zero for all 
H € P, we can consider this to be an example of Definition 6.1 by setting a(u, v; 4) := 
fa @eQuVu) - Vv dx and f(v;p) := lo qv dx. In the context of instationary two-phase 
flow, (6.2) needs to be solved in each time step for varying total mobilities (modeled 
by the parametric nature of x), while the permeability field x typically resolves fine 
geological structures and thus requires a very fine computational grid compared to 
the size of Q (see [90] and the references therein). 
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Example 6.3 (Incompressible fluid flow). The Stokes and Navier-Stokes equations 
represent a model of the flow motion for a viscous Newtonian incompressible fluid. 
In the steady case it can be formulated as follows: 


-vAy + 6(y-V)y+ Vp =f in Q, 


V-y=0 in Q, 

(6.3) 
y=8p on I’p, 
-pPn+ va = 8y onTy, 


where (y, p) are the velocity and the pressure fields defined on the computational do- 
main Q. The first equation expresses the linear momentum conservation and the sec- 
ond one the mass conservation, which is also called the continuity equation. Here f 
denotes a forcing term per unit mass, and gp and gy are the functions addressing the 
Dirichlet and Neumann boundary conditions respectively on Tp and Ty. The parame- 
terv = o/p denotes the kinematic viscosity, with p being the density and o the viscosity 
of the fluid. Navier-Stokes equations correspond to the case ô = 1; here we consider 
only 6 = O, the convective term is neglected, obtaining the steady Stokes equations, 
which provide a model in the case of slow motion of fluids with very high viscosity. 
We denote the functional spaces for velocity and pressure fields by X = (Hor, (Q))4, 
Q = L*(Q), respectively, where Hor, (Q) = {y € H'(Q) : ylp, = 0}. Moreover, for simplic- 
ity, we assume that gp = O (otherwise the lift function is required). The corresponding 
weak form of the Stokes equations (6.3) reads as follows: Find (y, p) € X x Q such that 


v | vy: vwda- | pv-waa= | f-wao+ | gy:war, VweX, 
Q Q Q Ty 


[av-yaa-o, V¥qeQ. 
Q 


(6.4) 


In a parameterized setting, the input-parameter vector y may characterize either the 
geometrical configuration or physical properties, boundary data, and sources of the 
problem. 

Denoting by V the product space given by V = X x Q, defining by u(w) = 
(y(u), p()) € V, and defining v = (w, q), the parameterized abstract formulation (6.4) 
can be rewritten in the following form: Find u(y) = (y(u), p(w) € V s.t. 


a(u(H), v;3H) =f(v;y) YveV, (6.5) 
where 
alu vip) =v | vy: vwaa- | pv-wao- | qv-y do, (6.6) 
Q Q Q 
fsp) = |f-wao+ | gy war, (6.7) 


Q Ty 


250 — A.Buhretal. 


Example 6.4 (Linear elasticity). We assume that Q c R? represents an isotropic homo- 
geneous material and we consider the following linear elastic boundary value prob- 
lem: Find the displacement vector u(y) and the Cauchy stress tensor o(u(j)) such that 


-V-o(u(u)) =G(w) in Q, 
o(u(u))-n=0 on Ty, (6.8) 
u(u)=8p on Tp, 


where the body force G : P — R? accounts for gravity. We can express for a linear 
elastic material the Cauchy stress tensor as o(u(u)) = E(u)C : e(u(y)), where C is 
the fourth-order stiffness tensor, e(u(u)) = 0.5(Vu() + (Vu(p))") is the infinitesimal 
strain tensor, and the colon operator : is defined as C : e(u(u)) = eet Cig€q(Uu(H)). 
Moreover, E : P — L” (Q) denotes Young’s modulus, which is assumed to be piecewise 
constant on Q and satisfy E(u) > Ey > O fora constant Ey € R*. Therefore, the stiffness 
tensor can be written as 


v 1 ete 
Cina = a F ví -— Z) ôijô + x0 a v) (64551 + 55x) 1< 1,); k, l< 3, 
where Ôij denotes the Kronecker delta; we choose Poisson’s ratio v = 0.3. The corre- 
sponding variational formulation of (6.8) then reads as follows: For any u € P find 
u(p) € V = {ve HQP : v=0on Tp} such that 


a(u(w),v;H) =f; Wwe. (6.9) 


Here, the bilinear and linear forms a(-,-;m) : [H'(Q))? x [H'(Q) > Rand f(;p) : 
[H!(Q)]? > Rare defined as 


ow' awk pa 
a(w, v; H) := [Ew S n and f(v;p) := | G(u) - v - a(G(p), v; p), 
a 4 a 


where Gu) c€ [H!(Q)]? denotes a suitable lifting function of the possibly nonhomoge- 
neous Dirichlet boundary conditions. 


To obtain approximate solutions of (6.1) we presume we have an appropriate 
grid-based numerical method at hand (the full-order model [FOM]), yielding a high- 
(but finite-)dimensional approximation space V,. We consider conforming continu- 
ous Galerkin (CG) FEs, where V, c V, and nonconforming discontinuous Galerkin or 
finite volume (FV) schemes, where V, ¢ V (in which case we require broken Sobolev 
spaces for our analysis, see Section 6.3.2.2). As a starting point for localized model re- 
duction we require the FOM space to be decomposable into “local” spaces, which we 
will make more precise shortly. While a localizing space decomposition could in gen- 
eral stem from any clustering of the degrees of freedom of V, (see for instance [24]), we 
are particularly interested in local approximation spaces which are associated with a 
domain decomposition of the physical domain. 
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Definition 6.5 (Nonoverlapping domain decomposition). We call a finite collection of 
M e N open polygonal subdomains Ty := 100%. ai pes a nonoverlapping do- 
main decomposition of the pbysital domain Q, if U Qm = Q and Qm N Qw = 9 
for1 < mm’ < M,m + m’. We collect in Ths T£, and 7 the sets of all vertices, 
edges, and facets (which we will denote interfaces from now on),! respectively, asso- 
ciated with the partition 7;, and define H := max”; diam Q. Moreover, we denote by 
r := U% 1 Qm) \ IQ the whole interface of the decomposition Ty. Note that Tj; = @ 
for d = 2 and T = TK = 0 for d = 1. Each of the sets Ty, Th» Th» and T% can be de- 
composed into lement associated with the domain boundary and inner elements, 
and we collect the latter in Th, TE, TE , and T , respectively. For instance, for each two 


adjacent subdomains Qm, Qm € Tp, there exists a shared interface Imm € Ty , while 
for all boundary subdomains Q, € Ty there exists at least one boundary interface 


Imao € TATZ. 


We can thus think of the domain decomposition as a usual grid, but without the 
requirements of 74 to actually resolve any data functions of the PDE. Given such a 
domain decomposition, we can abstractly define a localizing space decomposition. 


Definition 6.6 (Localizing space decomposition). Let the FOM space V, be a finite- 
dimensional Hilbert space with inner product and induced norm ||- l, = (, ye We 
call the direct sum decomposition of V, into subdomain spaces, interface spaces, edge 
spaces, and vertex spaces, 


n-e PY e @ v e @ Yk (6.10) 


YET ecTi veTy 
a localizing space decomposition. 


Note that such a decomposition is not unique and can always be found. Since the 
reduced space shall inherit this localizing decomposition, its purpose will be three- 
fold: (i) offline, it allows for an independent and localized generation of the local re- 
duced approximation spaces (compare Section 6.4), (ii) it allows to define and alter the 
physical domain Q online, given that local approximation spaces for certain reference 
subdomains have been prepared offline, and (iii) it allows to adapt a local approxima- 
tion space online (by adding basis functions or changing the local grid), while only re- 
quiring an update of local and neighboring prepared quantities (compare Section 6.6). 
For actual examples of space decompositions we refer to Section 6.3. 

Abstractly, we do not impose any further assumptions on the FOM as well as the 
reduced-order model (ROM). However, given the (bi)linearity of a and f, the compu- 
tational benefits of the localizing space decomposition are apparent (and are made 


1 Note that to simplify notation we denote both the upper bound of the continuity constant and the 
local interfaces with y, expecting that the respective meaning will be clear from the context. 
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more precise throughout the rest of this chapter). Since we allow for nonconforming 
approximations, in general we need to consider discrete counterparts of a and f which 
are only defined on the FOM space V, and not necessarily on V, where we again refer 
to the following sections for examples. 


Definition 6.7 (Locally decomposed full order model (FOM)). Let V;, be locally decom- 
posable as in (6.10), and let a,(-,-;M) : Vp x Vp > Rand f,(;M) € V} denote discrete 
variants of a and f, respectively, which are continuous and coercive with respect to the 
inner product of V,. For each y € P, find up(H) € V, such that 


apun, Vas) = fhn p) for all v, € Vp. (6.11) 


The idea of projection-based localized model order reduction is to consider a lo- 
cal reduced approximation space for each element of the localizing space decomposi- 
tion (6.10), in order to obtain a similarly decomposed reduced space Vy € V}: 


M 
Vy = Drv e Que BD ve GD vy. (6.12) 
m=1 


yeTz ecTi veTy 


with reduced subdomain spaces Vy c V;", reduced interface spaces Vi G vY , re- 
duced edge spaces Vx, c Vý, and reduced vertex spaces Vý c Vý. Similar to standard 
projection-based model order reduction, we obtain the ROM simply by Galerkin pro- 
jection of the locally decomposed FOM (6.11) onto this locally decomposed reduced 
space. 


Definition 6.8 (Locally decomposed reduced-order model (ROM)). Given a locally de- 
composed reduced space as in (6.12), for each u € P, find uy (yu) € Vy such that 


an(Uy() VNM) =fn(VnsH) for all vy € Vy. (6.13) 


The main questions remain: (i) how to choose good local reduced approximation 
spaces to guarantee accurate and at the same time efficient reduced-order approxima- 
tions, (ii) how to benefit from the localization of Vy, that is, how to obtain an offline- 
online decomposed scheme and in particular how to couple these local reduced ap- 
proximation spaces, and (iii) how to adaptively enrich these local reduced approxi- 
mation spaces online, if required. These topics will be answered throughout the re- 
mainder of this chapter, starting with examples of how to obtain localized FOMs from 
standard discretization schemes and how to couple the resulting local reduced spaces. 

Therefore, we introduce local grids T;,(Q,,) on each subdomain Qm € Tg, where we 
presume to resolve all data functions of the underlying PDE. As an analytical tool, we 
also define the global fine grid by T} = Uo, ¢7;, Th(Qm), which is usually not required in 
practical computations. For simplicity, we require the local grids of two subdomains 
Om Qn, € Ty to match along the shared interface Ymm € T and denote by T) Ymm) 
the corresponding set of all facets of T, which lie on Ym w. Finally, we require that T 
does not cut any grid cells. 
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6.3 Coupling local approximation spaces 


6.3.1 Conforming approach 


There are various ways to couple local reduced spaces such that we obtain a conform- 
ing approximation, such as the partition of unity method [11] or the GFEM [8, 7, 11, 10]. 
However, in this section we focus on the decomposition into interface spaces and 
intra-element spaces, where the coupling is performed via the coupling or interface 
modes spanning the interface space. 


6.3.1.1 The multidomain problem and the Steklov—Poincaré interface equation 


First, we introduce local Hilbert spaces H} (Qm) c V” c H (Om), m = 1,...,M, which 
are supposed to respect the boundary conditions on 0Q, the local spaces V{!' := {v € 
vV™ : v| ag,,\aa = O}, and the trace space A associated with T. Moreover, we introduce 
local parameter-dependent bilinear and linear forms a,,(-,-;H) : V™ x V” > Rand 
fap) € V™, y € P,m = 1,...,M, and the inner product (.,-)yn : V"x V” > R. 
We may then state the variational form (6.1) equivalently as follows (see for instance 
[98]): For any u € P find u,,(u) € V™, m =1,...,M such that 


amum), V3 H) = fav; y) W € Vg’, (6.14a) 
Um (H) = Um (W) on Tmm> (6.14b) 
M M 
Y Am(Um (H), Em63 H) = $ SnlEms H) YỌ €A, (6.14c) 
m=1 m=1 


where En : A > V™, m = 1,...,M, are linear and continuous extension operators. 

The formulation (6.14) can then be used to derive an equation that solely acts on 
functions on the interface but nevertheless uniquely defines the solution u(y) of (6.1). 
To that end, we introduce a parameter-dependent lifting operator €>_, (uM) : A > 
V, where €r_,o(H)¢ is defined as the minimizer of inf,)<y a(v(H), v9); p) subject to 
v(H)|r = ¢. Note that we then also have 


amlEr oW vy) =0 VWeV,' and EMF =f on TNA ,. (6.15) 


Then, we can rewrite the solution u(y) as 


M 
UCH) = Ero DUW) + > wh, (6.16) 


m=1 
where uf (p) € Vi" solves 


Am (Ui, (HM), V3) =fin(Viw) YW eV”, m=1,...,M. (6.17) 
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Inserting (6.16) into (6.14c) and choosing Em = Eso, (H) yields the Steklov—Poincaré 
interface equation: Find u()|; € A such that 


M 
Y. am(Ero, () (Ur), Ero, (SB) 


3 
T, 


(6.18) 
M 
= X [fmlErso,, GDH) - amh (Ws Ero, (WSsw)] YE EA. 


m=1 


Let us note that the Steklov-Poincaré interface equation and its discrete, algebraic 
analogon, the Schur complement system, are at the base of iterative substructuring 
methods (see [98, 110]), which have been combined with the reduced basis method 
in [80]. 

We may finally define a space associated with the interface V” as V" = {Erno € 
V :¢ € A} and obtain the decomposition V = (@"_, Vi") e V". This decomposition is 
a-orthogonal thanks to (6.15). 

While the computation of the (harmonic) lifting operators is inherently local 
(see (6.15)), the Steklov-Poincaré interface equation is posed on the whole interface 
T. To localize the latter we decompose V! as we will describe next. 


6.3.1.2 A conforming, localized reduced-order approximation 


First, we determine basis functions associated with the vertices v € Ty . One common 

approach [52, 64, 20] is to require that a basis function Y“ € V} N [HA ( Um Omz = 
VCO 

1,...,d, associated with some vertex v of the coarse mesh Ty satisfies for all Q, m = 

1,...,M, 


(p’,w)ym=0 VweVpy and p'(x") =1, p(x”) =0,vev. (6.19) 


Here, x” are the (global) coordinates of the vertex v and Vo := {V € Vj": v = 0 on aQ,,\ 
Ty}. To uniquely define Y“ we need to prescribe the respective values on T. We may for 
instance require Y” to be linear on the respective edges or bilinear on the respective 
interfaces (see, e. g., [20]). For multiscale problems in two space dimensions with a 
permeability x(x,, X>;}1) it has been suggested in [54] to prescribe 


X1 


v vy. ds F ds 
Y“ (XX3) := ( | ER Vf | To <a) (6.20) 


1 1 


I 
on a horizontal edge [x] , x7] x {x5} in a uniform rectangular coarse grid Ty. 
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; ; ; ; Nf 
Next, we assume that we have given sets of discrete edge basis functions {y;},.") € 


V,,|- and discrete interface basis functions one € Val, defined on the respective edge 
e € Ty or interface y € 7}. Here, we set Ngo = dim(Vple\ge) and Nj, = dim(Vj,|,\ay) as 
we require that yy and x are zero on the boundary of the edge and interface, respec- 
tively. Furthermore, we define Ane, := span{yt,..., XNE) and Ajy := span{y},..., Xn) 

Similarly as for the vertices we may then define associated basis functions that 
have support on the union of subdomains that share the respective edge or interface: 


Find wy € V [Ho( Um Om)’, Z = 1,...3d, y € Ti k = 1,...,Nj.os such that 
YCOm 
(WLW)ym =O WwevVy and wily =x}. (6.21) 


Likewise, we find py € Vp N ([Ho(Um Om), Z = 1,...,d, € € TK, k = 1,- -> Njo» such 
ecOm 


that 
(Wi.W)ym=O0 YweVro and wile =X¢- (6.22) 


Again, we need to provide the value of Yẹ on the interfaces sharing the edge e € Tj in 
order to uniquely define ij. Similarly to above we may require that wy is linear on the 
respective interfaces as suggested for instance in [20] or define a function which takes 
into account also the coefficient function. 

Note that if the interfaces are mutually disjoint, which is for instance the case if we 
associate the subdomains Qm, m = 1,...,M, with the components of a structure, only 
the basis functions YY, k = 1,..., NY o (d = 3), or py, k = 1,.. ., Ngo (d = 2), are needed. 
Here, the values of the basis functions on the boundary of the interfaces or edges are 
determined by the boundary conditions on 0Q (see for instance [61, 60, 37, 103]). 

For NY « Ns and N° « Ngo we may now define the reduced space associated 
with T as follows: 


V} = @® span{p’} e B spanfyi,.... Wye} e  span{py,...,Wh,}. (6.23) 


veTy ecTi yeTy 


Such reduced interface spaces are for instance employed in (adaptive) CMS [52, 64], 
the scRBE method [61, 60, 37, 103] for mutually disjoint interfaces, or in the ArbiLoMod 
[20]. Subspaces of es are considered in certain multiscale methods. For example, in 
the MsFEM of Hou and Wu [54], the reduced space is spanned by the basis functions 
Wve Ty . For further relations between CMS, the MsFEM, and the GFEM we refer, 
e. g., to [52]. 

Recall that the basis functions associated with the vertices, edges, and interfaces 
have all been computed with respect to an inner product that does not depend on the 
parameter (see (6.19), (6.21), and (6.22)). Therefore, we finally assume that we have also 
given reduced spaces Vyy.o := span{(j",..., Gym} C Vino m = 1,...,M, that will account 
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for parameter variations. In detail, we obtain approximations Wy (p), * = v,e,y by 
solving 


find by (p) € Vaio! %m(by + bym, w;u)=0 vwe VNo (6.24) 


and setting ý (u) = Yk + by}, * = v,e,f. Finally, we define b”(u) € Vio as the 
solution of 


find b” (u) € VNo: am(b" (u), w;p)=f(w,p) Ywe Vio: (6.25) 


Note that both b; (u), « = v, e, y and b™ (p) can be interpreted as intra-element reduced 
basis approximations; compare to Chapters 1 and 4 of this volume of Model order re- 
duction. The corresponding reduced spaces VNo» m = 1,...,M, can for instance be 
constructed from solutions b; (y), b™ (p) € Vro» * = v,e,f, of 


am(pk + bym) w;p)=0 Yw € Vio (6.26) 
and 
am(b” (4), Ws) =f(w,p) Yw € Vio (6.27) 


respectively, via a standard greedy algorithm or a POD.? Let us also remark that for 
instance in the scRBE method for the approximation of each basis function pý, * = 
v, e, y, a different reduced basis space is considered, to further reduce the size of prob- 
lems (6.24), (6.25). Finally, we define the reduced spaces 


M 
Vy = BD Vio ® VN (6.28) 
m=1 
and 
VH) = QB span" w} © GD span{py@),.... Pre(w} 


veTy ecTy 


® B span{py (u),..., y D}. 


YeTy 


(6.29) 


The global reduced approximation uy(m) can then be computed by performing a 
Galerkin projection onto the reduced space Vy (p) or a Petrov-Galerkin approxima- 
tion using Vu) as a trial and VE as a test space (see, e.g., [38, 100]). Instead of 
eliminating the volume degrees of freedom via (6.24), (6.25), uy(j) can also directly 
be determined by performing a Galerkin projection onto Vy (see for instance [20]). 


2 Note that in actual practice one would construct the reduced bases only on a certain number < M 
of reference domains; see for instance [61]. 
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Similarly, for CMS and a fixed parameter a Galerkin projection onto Vy may be per- 
formed to compute the reduced solution; here, the reduced space Vio is constructed 
from an eigenvalue problem and does not account for parameter variations (see, e. g., 
[52]). Finally, in the reduced basis-domain decomposition—FE (RDF) method [63] the 
reduced space Vy is chosen as a direct sum of Q% Vy;o and standard FE spaces 
defined on the interface or on a (small) area around the interface. Here, the intra- 
element reduced spaces Vo are constructed via a greedy algorithm considering a 
parameterized linear combination of standard Lagrange basis functions or Fourier 
modes as boundary conditions. Then, a Galerkin projection on Vy is performed to 
compute uy(Ņ). 


6.3.2 Nonconforming approach 


With the term nonconforming approach we want to classify a set of alternative tech- 
niques to solve the reduced problem on the global computational domain. A first ap- 
proach consists in considering a global system of equations given by local parameter- 
ized problems and additional equations ensuring the matching between the different 
subdomains through the use of Lagrange multipliers. This approach has been used 
for solving elliptic equations in [76, 77] and Stokes equations in [74, 62]. 

Another approach consists in coupling local FOM spaces by interior penalty (IP) 
bilinear forms, inspired by discontinuous Galerkin FEM. Here, we refer to the discon- 
tinuous Galerkin RBEM [27, 6, 93] and the local reduced basis discontinuous Galerkin 
approach [66]. A discontinuous Galerkin approach with local POD modes has been 
presented in [41]. In the context of multiscale problems (cf. Example 6.2), the gener- 
alized multiscale discontinuous Galerkin method has been proposed in [29, 30] and 
used for solving the heat problem with phase change in [107]. We will present the 
LRBMS method in Section 6.3.2.2. LRBMS has been introduced in [5] and analyzed in 
[89, 90] for elliptic and in [87] for parabolic problems. Applications to the simulation of 
two-phase flow in porous media have been addressed in [65] and to battery simulation 
with resolved electrodes in [86]. 


6.3.2.1 Nonconforming coupling based on Lagrange multipliers 


We want to reformulate the problem (6.14), with the idea that exact coincidence of 
the traces of the discrete functions (equation (6.14b)) is generally too stringent, and 
may, in fact, lead to imposing um = 0 on the internal interfaces; thus, the gluing pro- 
cess can be done in a dual way through Lagrange multipliers. We assume that local 
basis functions are computed in each subdomain Qm, M = 1,..., M, by solving local pa- 
rameterized variational problems coming from the original problem (6.1) with proper 
boundary conditions along the boundaries which correspond to internal ones in the 
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original domain. The choice of the boundary conditions is strongly related to the prob- 
lem aimed to be solved. Thus, local reduced spaces are defined via these local solu- 
tions and denoted by V}, m = 1,...,M. Possible ways to construct Vj’ are presented in 
Section 6.4. If two or more subdomains are characterized by the same type of param- 
eter and the same type of boundary conditions, the same local reduced space can be 
associated to those subdomains. For simplicity we consider different spaces for each 
different subdomain. 
We define the following operator: 


c™™" (u(y), Y) = | (ula, -Ulo ds = 0, Vip € Wmm» (6.30) 
Din! 

where m,m’ € {1,...,M}, Tmm is the interface between two adjacent subdomains de- 
noted with the indices m and m’, respectively, and Wmm is the Lagrange multiplier 
space defined on this interface. Typical choices for the latter are low-order polynomial 
spaces [76, 62] or spaces constructed from snapshots (and their derivatives) [77]. 

A basis for Wm, can then for instance be provided by the characteristic Lagrange 
polynomials %4, 4 = 1, .. . , Qm w» associated with the Qm, w nodes Of T mw- 

If we suppose that Q has M - 1 internal interfaces, I, »4;,m = 1,...,M — 1, the 
reduced global problem of this approach reads as follows: Find uy(p) € Vy XX Vi , 
Ay € Wmm M =1,..., M - 1, such that 


M-1 
a(Uy(H), Vy» H) + $ L vy Ay) = f(W, Wy € Vy x- x VA, E 
i=l : 


Lm (uw), p) =O m=1,...,M-1,VY € Wmm: 


6.3.2.2 Nonconforming coupling based on interior penalties 


We demonstrate how to obtain a localized FOM by applying ideas from IP discontin- 
uous Galerkin schemes with respect to the domain decomposition 7y in the context 
of the parametric multiscale Example 6.2. To define the localized FOM, we presume 
we are given a discretization on the full global grid T, (which is not used in actual 
computations), which we make precise by specifying the approximation space with 
an associated inner product and discrete variants of a and f. As a common ground 
for the analysis of conforming as well as nonconforming schemes, we introduce the 
broken Sobolev space H*(t;,(w)) := {v € L? (w) | vle € H°(t) Vt € T,(w)}, for a given 
grid T, (w) of some domain w c Q and s > 1, and associated broken gradient operator 
Vp: H' (t (w)) > L’w)4 by (Vpv)le := V(vle) on all t € T for v € H'(t),(w)). 


Example 6.9 (Continuous Galerkin (CG) FEM). The CG FEM scheme for the conforming 
approximation of Example 6.2 with respect to the full global grid T, is given by the 
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conforming approximation space of order k > 1, 
Vr o(th) = {v € V | vle € P(E) Vt € Tp} € V c H(t), 


where P(w) for any w c Q denotes the space of all polynomials defined on w of degree 
at most k > 0; the bilinear form (.,-)°° : H\(r),) x H!(T}) > R, given by (u, v)® := 
fo Vnu: Vpv dx, as the inner product on VEE (Th) (where we note that its restriction to V c 
H ler) coincides with the V-inner product); and the discrete bilinear form arS(., sH): 
H' (Tp) x H' (Tp) > R and linear functional f° : H'(T,) > R, given by 


ay°(u, v; M) := | eanv,u) -V,vdx and fw) = [avax 
a a 


(again noting that their respective restrictions to V coincide with a and f). 


The definition of the nonconforming scheme is more involved. We denote the set 
of faces of T, by T) and to each face ø € Tis we assign a unique normal n, € RÊ 
pointing away from t*, where the face may be either an inner face ø € Ths given by the 


intersection of two grid elements t*, t € Tp, 0 = tt Nn t~, or a boundary face o € tT’, 


given by o = tt N dQ for some t* € Tp. Since functions in the broken Sobolev space are 
two-valued on grid faces, we introduce the mean (-) and jump [-] on a boundary face 
by (v) := [v] := v| and by (v) := ile +v|+-)and [v] := vle - v|}, respectively, on any 
other face. 

Considering the family of IP discontinuous Galerkin schemes, we present the sym- 
metric variant for ease of notation, and refer to the symmetric weighted variant [40], 
which is particularly well suited for multiscale problems with highly varying or high- 
contrast coefficients. 


Example 6.10 (IP discontinuous Galerkin FEM). The symmetric IP discontinuous Ga- 
lerkin FEM scheme for the nonconforming approximation of Example 6.2 with respect 
to the full global grid T, is given by the nonconforming approximation space of order 
k>1, 


VPS (Th) = fv € LQ) | vle € P(t) Wt € Tp} c A" (tp); 
the bilinear form (., ape : H' (T) x H' (Th) — R, given by 
(u,v) = (uvt Y uvi with (u,v) = ac [v] ds, 
oer), o 


as inner product on vP G(T), where h, is a positive number associated with each face 
o E€ Trs e.g., h, := diam(o) for d > 2and h, := min{diam(t*), diam(t~)} for d = 1; 
and the linear functional fe C.H Htp) — R given by fP s (v) := iW) and the discrete 


bilinear form apts Sp): H’ (Th) x H? (Tp) — R, given by 


aps (u, v; H) := ay(u, V; PL) + > ag (U, v; H) 
oet) 
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with the face bilinear form a, for any ø € T) given by 
,(V, W; H) := aC (v, u; H) + af (u, v; u) + (U, VÈ Wg 


with af (u, v; 4) := Í -((K(u)Vpv) - no) [u] ds and a user-dependent penalty weight w, > 
0, such that a is continuous and coercive with respect to the above inner product. 


The main idea of an IP localized FOM is to consider the restriction of either of the 
above discretization schemes to each subdomain of the domain decomposition, and to 
again couple those with IP techniques along the interfaces of the subdomain. We thus 
choose « € {CG, DG} and obtain the localized FOM space in the sense of Definition 6.6 
as a direct sum of subdomain spaces (with empty interface, edge, and vertex spaces) 


M 
Va = V, with Vi" = {vlo, |v € V*}, 


m=1 


with associated inner product (.,-)y. : Vp x Vp > R given by 
Vp: “aX Vh 


M 
(uv)y, = È UlopYlo + È > uv 
m=1 


eT? oer, (T') 


We also define the linear functional f} : Va —> R by fy := fẹ and, in a similar manner 
as above, the nonconforming bilinear form ap(.,-;) : Vp x Vh > R by 


M 
ay, (U, v; H) = Y ay lo, Yla; W) + $ $ aç(u, v; y). 
m=1 


PeT? oer; (T') 


We have thus fully specified a localized FOM in the sense of Definition 6.7 and com- 
ment on two special cases: for + = CG and a trivial domain decomposition of a single 
subdomain, Ty = {Q}, we obtain the above standard CG FEM, while for « = DG the re- 
sulting FOM coincides with the above standard symmetric IP discontinuous Galerkin 
FEM. 

To make the coupling more precise, we may rearrange the above terms to obtain 
a localization of a; with respect to the domain decomposition in the sense of 


M 
auvi) = X apu vy)+ È a (wv), 
m=1 PeT} 
with the subdomain and interface bilinear forms 


ap (wv; H) : = ap Ulo p Vlo D+ Y È asla, Vlo, M) 


T ETZ NOn oer} (I") 
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Y {asula Vlos H) + aglulo vlos W} 


y 
oet I) 


al (u, v; H): 


respectively, for all1 < m < M and all T' € T}, with the subdomains Q*, Q7 € Ty 
sharing the interface T”. 

Now given a local reduced space Vy c V;" for each subdomain we obtain the 
locally decomposed broken reduced space in the sense of (6.12) by 


M 
Vy = QD Vn c Vre 


m=1 


Using the above decomposition of a, into subdomain and interface contributions, we 
can readily observe that the locally decomposed ROM can be offline-online decom- 
posed by local computations, namely, by projection of the subdomain bilinear forms 
a’ (.,; p) onto VË x V” and the interface bilinear forms af (., -; u) onto V® x Vi, with 
1<m,n< M, such that Q* = Qm and QO” = Q,, respectively. 

We thus obtain a sparse matrix representation of the resulting reduced system, 
with a sparsity pattern which coincides with the one from standard IP discontinuous 
Galerkin schemes. 


6.4 Preparation of local approximation spaces 


Both couplings that yield a conforming and nonconforming approximation require ei- 
ther reduced spaces Ny c Valy for interfaces and/or edges Ajye € Vrle (Section 6.3.1) or 
reduced spaces Vý (Section 6.3.2) or both. As the generation of edge basis functions 
can be done analogously to the construction of interface basis functions we restrict 
ourselves to the latter in order to simplify notation. To fix the setting we thus consider 
the task of finding a suitable reduced space on either a subdomain Qm ¢ Qout € Q 
with dist out Qm) 2 P > 0, Tout = AQAgye \ OQ or an interface Ty, 7 C AQ., where 
dist outs lmm) 2 P > 0. Possible geometric configurations of the oversampling do- 
main Q,,, are illustrated in Figure 6.1. 


AQ 
I oy te aut Om | Foe Figure 6.1: Illustration of possible decompositions of Oout with 
oQ respect to Tnm OF Qm. 


We will first briefly discuss in Section 6.4.1 reduced spaces that are spanned by polyno- 
mials or solutions of “standard” eigenvalue problems and are thus related to the spec- 
tral element method or hp-FEM. Subsequently, in Section 6.4.2 we will present reduced 
spaces that are generated from local solutions of the PDE, are thus of empirical nature, 
and are optimal in the sense of Kolmogorov. We will also show how those optimal basis 
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functions can be efficiently and accurately approximated by means of random sam- 
pling. 


6.4.1 Polynomial-based local approximation spaces 


CMS as introduced in [59, 12] relies on free vibration modes or eigenmodes of local, 
constrained eigenvalue problems [59, 12, 17, 52, 64, 51] for the approximation within 
subdomains. To couple the modes at the interfaces a reduced interface space spanned 
by eigenmodes is employed [59, 12, 17, 52, 64, 51]. 

A combination of domain decomposition and reduced basis methods has first 
been considered in the RBEM [76]. Here, inspired by the mortar spectral element 
method [16], the Lagrange multiplier space Wim! as defined in Section 6.3.2 is chosen 
as a low-order polynomial space. The reduced basis hybrid method [62] extends the 
RBEM by additionally considering a coarse FE discretization on the whole domain to 
account for continuity of normal stresses and also employs a low-order polynomial 
Lagrange multiplier space on the interface. For the scRBE method a reduced inter- 
face space spanned by the eigenvectors of a discrete generalized eigenvalue problem 
based on the Laplacian has been suggested in [61, 60] and eigenmodes of a singular 
Sturm-—Liouville eigenproblem have been used in [37]. Finally, in the RDF method [63] 
a standard FE space is considered on the interface or on a (small) area around the 
interface. 


6.4.2 Local approximation spaces based on empirical training 


In this subsection we are concerned with local approximation spaces that are con- 
structed from local solutions of the PDE; those approaches are often called empiri- 
cal. Basis functions on the interfaces selected from local snapshots are for instance 
suggested in [37], where an empirical pairwise training procedure for interface re- 
duction within the scRBE context is developed, and within a heterogeneous domain 
decomposition method in [83]. Local approximation spaces that are optimal in the 
sense of Kolmogorov have been introduced for subdomains within the GFEM in [10] 
for parameter-independent PDEs and for interfaces within static condensation pro- 
cedures [103] for parameterized PDEs. While the authors of [103] introduce and ana- 
lyze a spectral greedy algorithm to deal with parameter variations, [109] suggests us- 
ing a POD making use of the hierarchical approximate POD [53]. Those optimal local 
spaces both allow for a rigorous a priori theory and yield a rapidly (and often exponen- 
tially) convergent approximation; in certain cases the superalgebraic convergence can 
be proved [10]. Recently, in [109, 108] the results in [10, 9, 103] have been generalized 
from linear differential operators whose associated bilinear form is coercive to elliptic, 
inf-sup stable ones. In [21] it has been shown that those optimal local approximation 
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spaces can be efficiently approximated by transferring methods from randomized nu- 
merical linear algebra [46]; the local approximation spaces in [21] are constructed from 
local solutions of the PDE with random boundary conditions. Local reduced spaces 
generated from random snapshots have also been suggested in [20, 37] and methods 
from randomized linear algebra have been exploited in the FETI-2A domain decompo- 
sition method in [113] and in [23] for the generalized multiscale FEM. 

We will first present the optimal local approximation spaces as introduced in [10, 
103] for a fixed parameter P = {ji}, subsequently discuss their approximation via ran- 
dom sampling, and conclude this subsection with the discussion of the general case 
P + {p}. To simplify notation we will omit pas long as it is fixed. 


6.4.2.1 Optimal local approximation spaces for P = {ji} 


To enable maximum flexibility regarding the shape of Q on the user’s side, we assume 
that we do not have any a priori knowledge of the shape of Q when constructing the 
ROM. We thus know that the global solution u satisfies the considered PDE locally on 
Qout but suppose that the trace of u on 0Q,,; is unknown to us. Therefore, we aim at 
approximating all local solutions u,,, of 


loculo V) = floc(V) WV € Vioo (6.32) 


with arbitrary Dirichlet boundary conditions on T „ut. Here, the Hilbert space Vioc is de- 
fined such that [H}(Qout)]" € Vioc € [H'(Qout) 7, Z = 1,..., d, respecting the boundary 
conditions on 0Q, and djoe : [H!(Qout)]7 x [H (Qout)] > R; fioc : Vioc > R are local 
bilinear and linear forms. We will first restrict ourselves to the case fioc = 0, 8p = 0, 
and 0Q,,, NTp = 9; the general case will be dealt with at the end of this subsection. We 
may then define the space of all local solutions of the PDE as 


H := {w € [H'(Qout)] : w solves (6.32),w =OonTpNAQ nt}, Z=1....d. (6.33) 


As suggested in [10, 103] we introduce a transfer operator T : S — R for Hilbert 
spaces S and R, where S = {wlr a : W € H}. We define 7 for interfaces or subdomains, 
respectively, for w € H as 


T(wlr.,,) = (w -= Po, (w)) 


out 


or Twr) = (w - Po (w))lo, (6.34) 


Dum! 


and set R = {vlr „ : V = W- Po, (w),w € H}orR = {(w- Po wilo, : W € H} 
Here, Pp, D c Qo,;, denotes an orthogonal projection onto the kernel of the bilinear 
form; for further details see [21, 103]. In the case of heat conduction we would for in- 
stance subtract the mean value of the respective function on D. Note that subtracting 
this projection is necessary to prove compactness of the transfer operator 7. The key 
argument to show compactness of 7 is Caccioppoli’s inequality, which estimates the 
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energy norm of a function in H on Q, in terms of the L?-norm on Q,,; of the respec- 
tive function. Using the Hilbert-Schmidt theorem and Theorem 2.2 in [95, Chapter 4] 
it can then be shown that certain eigenfunctions of 7*7 span the optimal local ap- 
proximation space, where 7* : R — S denotes the adjoint operator of T. As we aim 
at approximating H and thus a whole set of functions, the concept of optimality of 
Kolmogorov [68] is used: A subspace R,, c R of dimension at most n for which holds 


ITY- Cle 
= f 
STO) = AP a Tih. 


is called an optimal subspace for dp (T(S); R), where the Kolmogorov n-width 
d (T(S); R) is defined as 


d,(T(S);R) = inf sup inf Slr, 
RryrcR pes CERn Ills 
dim(R,,)=n 
We summarize the findings about the optimal local approximation spaces in the 
following theorem. 


Theorem 6.11 (Optimal local approximation spaces [10, 103]). The optimal approxi- 
mation space for d,(T(S);R) is given by 


Rn i= spani >- Xn} where X? = Toj j =1,...n, (6.35) 


and À; are the largest n eigenvalues and ġ; the corresponding eigenfunctions that satisfy 
the following transfer eigenvalue problem: Find (9;,A,) € (S, IR*) such that 


Moreover, we have 


ITE Cle 
U RE int ae = Viet en 


Remark 6.12. We emphasize that the optimal space R, is optimal in the sense of Kol- 
mogorov for the approximation of the range of 7 and not necessarily for the approxi- 
mation of u(y). Moreover, we remark that X are the left singular vectors and yA; the 
singular values of T. 


Next, for fioc + O but still gp = O we solve the following problem: Find ul se 
such that docli oY oe Y) = fioc) for all v € Vioc and augment the space R, with either 
ul ocla,, OF w lr oclT PE A take nonhomogeneous Dirichlet boundary conditions into ac- 
count one can ‘proceed for instance with a standard lifting approach, adjusting fioc 
accordingly. Note that for homogeneous boundary conditions we proceed very simi- 


larly to above, prescribing “arbitrary” boundary conditions on Tut and homogeneous 


€ Vioc 
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boundary conditions on ðQ N ðQ ut- The optimal local approximation spaces for sub- 
domains are then defined as 


Ry = span{y,”,....x2”, uf elo, } @ ker(a;,(-,v)) (6.38) 
and similarly for interfaces as 
Ry, = span{y;”,....X;?, loclt, y} ® ker(am(+¥))Ir,.- (6.39) 


Here, ker(a,,(-, v)) denotes the kernel of the mapping a,,(.,v) : [H KOMIK > Rz = 
1,...,d, v € Ves for the bilinear form a,,, defined in Section 6.3.1. In the case 0Q,,,NT'p + 
ø, all modifications in this subsection involving the kernel of the bilinear form are 
waived. 

The result in (6.37) can be exploited to derive an a priori error bound for the ap- 
proximation error between the solution u(j) of (6.1) still for a fixed reference parameter 
jÑ and the optimal static condensation approximation u” (ñ) as stated in the following 
proposition. 


Proposition 6.13 (A priori error bound [103]). Assume that the interfaces y € Ty are 
mutually disjoint, that all interfaces have the same geometry, and that each Qm, m = 
1,...,M, has exactly two interfaces. Let u(ja) be the (exact) solution of (6.1) for a fixed 
parameter jt. Moreover, let u,, (ft) be the static condensation approximation defined in 
Section 6.3.1, where we employ the optimal interface space R} for each y € Ta and as- 
sume that the error due to the intra-element reduced basis approximation is zero. Then, 
we have the following a priori error bound: 


lu) = un, la < #y max(C, VAX 1) (6.40) 


lulia yer? 


where #y denotes the number of interfaces in T) and À% is the (n + 1)-th eigenvalue 
of (6.36) for the interface y € ea The constant C, depends only on the subdomains that 
share the interface y and neither on Q nor on u(jt). 


To define reduced interface spaces Ny» Ye, Y, and reduced spaces Vi", m = 


1,...,M, we approximate (6.36) with FEs. To that end, we introduce a conforming FE 
space Vpigc © Vioc, the FE source space S := {vlr „ : v € Vn} of dimension Ns, and 
the FE range space R := {(v - Poga MIT :v € V,}orR := {(v - Po lo, : v € Vz} 
with dim(R) = Np. We may then define the discrete transfer operator T : S — R for 
w € Hp = {W € Vrlo u © ocw P) = OVP € Vagos W=0 on Tp NM OQoyt} as 


Twr a) = (w - Po,,, (W)Ir n or T(wly,,.) = (w - Po, (w))lo, - (6.41) 


In order to define a matrix form of the transfer operator we introduce degree of free- 
dom mappings Bs_,y e REMV lous and By p € RMN low) that map 
—Vnloout hlaou ™ 
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the degrees of freedom of S to the degrees of freedom of V} lo, „ and the degrees of free- 
dom of V;,|9,,, to the degrees of freedom of R, respectively. Moreover, we introduce the 
stiffness matrix Ajoc obtained from the FE discretization of (6.32), where we assume 
that in the rows associated with the Dirichlet degrees of freedom the nondiagonal en- 
tries are zero and the diagonal entries equal one. By denoting by ¢ the FE coefficients 
of ¢ € S and by defining Pp as the matrix of the orthogonal projection on the ker- 
nel of the bilinear form on D c Q,,;, we obtain the following matrix representation 
T € RA®*s of the transfer operator for subdomains 


-1 
TÒ = (1 = Po BV plage RA Bs Vila Ç (6.42) 


and interfaces 
-1 
TÒ = Bylo RC = Po. JA Bs Vila Ç. (6.43) 


Finally, we denote by Ms the inner product matrix of S and by Mp the inner product 
matrix of R. Then, the FE approximation of the transfer eigenvalue problem reads as 
follows: Find the eigenvectors ¢; € R^S and the eigenvalues A; € Ro such that 


The coefficients of the FE approximation of the basis functions {y,”,...,X;?,} of the 
discrete optimal local approximation space 


Ry = span {ps oX pa (6.45) 


are then given by Xn =T¢ pj=1,...;n. Adding the representation of the right-hand 
side, the boundary conditions, and a basis of the kernel of the bilinear form yields the 
optimal spaces A% and Vj". 

Note that in actual practice we would not assemble the matrix T. Instead one 
may solve the PDE locally Nz times prescribing the basis functions of S as Dirichlet 
boundary conditions on Tout and subsequently assemble and solve the transfer eigen- 
value problem. Alternatively, one may pass T implicitly to the Lanczos method. For 
instance, the implicitly restarted Lanczos method as implemented in ARPACK [73] re- 
quires O(n) local solutions of the PDE in each iteration and applications of the ad- 
joint T*. In the next subsection we will show how methods from randomized linear 
algebra [46, 32, 78, 79] can be used to compute an approximation of the optimal local 
approximation spaces. However, beforehand, we conclude this subsection with some 
numerical experiments on the transfer eigenvalues and thus via Proposition 6.13 on 
the convergence behavior of the relative approximation error. 

To this end, we present the simplified model for a ship stiffener from [103]: We 
consider Qout = Q4 U Ô, and Vin mt = T12 = O, N Q5, where Q; is depicted in Figure 6.2, 
Q; is just a shifted version of Q,, and the part of Fout in Q, is indicated in yellow in 
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Cmm 


part of 


Pout 


x 


Figure 6.2: Mesh in the subdomain Q; for the ship stiffener. The part of Tout in Q3 is indicated in 
yellow and on the opposite we have the interface F m- In the red shaded areas Young’s modulus 
may be varied between 1 and 20 and in the gray areas we consider E(u) = 1. 


Figure 6.2. We allow E(u) to vary in the red areas of the subdomains between 1 and 20 
and prescribe E(u) = 1 in the gray areas; we choose G(p) = (0,0, o)". 

In detail, we consider Q, = (—0.7, 0.7) x (—0.05, 0.05) x (—0.6, 0.6), Q, = (0.7, 2.1) x 
(-0.05, 0.05) x (-0.6,0.6), and Pout = {-0.7} x (-0.05, 0.05) x (—0.6, 0.6) U {2.1} x 
(-0.05, 0.05) x (—0.6, 0.6). We employ a conforming linear FE space associated with 
the mesh depicted in Figure 6.2, resulting in N = 13,125 degrees of freedom per sub- 
domain and an FE interface space of dimension Np = 375. Finally, we equip both S 
and R with a lifting inner product based on the lifting operator €;_,9 (j) defined in 
Section 6.3.1.1; for further details we refer to [103]. 

We consider different values for Young’s modulus (ratios) Ei ,i = 1,2, in the red 
areas of the subdomains and observe in Figure 6.3 for the ship stiffener application 
an exponential convergence of order = e™” of the eigenvalues À„(4) and thus the static 
condensation approximation. We emphasize that we observe in Figure 6.3 that the 
eigenvalues associated with the stiffened plate (Ej = E} = 20) decay fastest, while we 
see the slowest decay for the nonstiffened plate (Ej = E} = 1). This is consistent with 
the expectation that stiffening the plate decreases the deflection of the plate, eliminat- 
ing the higher eigenmodes. Moreover, an inspection of the optimal interface modes re- 
veals many “classical” mode shapes such as bending or torsional modes of beams and 
demonstrates again the physical significance of the optimal modes. Also for beams of 
different shapes, including an I-beam with a crack and thus an irregular domain, an 
exponential convergence of the transfer eigenvalues and the physical significance of 
the transfer eigenmodes can be observed; for further details see [103]. 


Ej =E; =1 
Ej = E} = 20 
E =1;E} = 20 
E = 10; E = 13 
0.01e” 


Figure 6.3: Eigenvalues A, (4) for 


I EET i a T a ain different Young modulus ratios E; 
n in Q; i= 1,2. 


268 —— A.Buhretal. 


6.4.2.2 Randomized training 


In order to compute an efficient approximation Rand of R,, the adaptive randomized 
range approximation algorithm, Algorithm 6.1, as suggested in [21] iteratively en- 
hances the reduced space with applications of T to a random function until a certain 
convergence criterion is satisfied. 


Algorithm 6.1: Adaptive randomized range approximation. 
Input : Operator T, target accuracy tol, number of test vectors n,, maximum 
failure probability £a1gofail 
Output: space R"* with property P(|IT - Prana T|| < tol) > (1 ~ Eaigofai) 
1 Initialize: B — 0, M — {TD,"rj, ..., TD;'r,,} 
2 Compute error estimator factors: 


Ms 


Etestfail T Ealgofail/Nr3 Cest — [2aMs erf 1( "/Etesttail)! 
while (max;¢y lltliR) - Cest > tol do 

B — Bu (TD;'r) 

B — orthonormalize(B) 

orthogonalize test vectors: M — {t — Popanat | t € M} 


N Au A W 


8 return pend = span{B} 


In detail, in each iteration on line 5 we draw a new random vector r € RNs whose 
entries are independent and identically distributed random variables with standard 
normal distribution. Then, we employ the mapping De : RNs — S to define a unique 
FE function in S whose coefficients are the components of r. Subsequently, we apply 
the transfer operator T to D;'r, meaning that we solve the PDE locally on Q,,; with 
random boundary conditions and restrict the solution to Qm or Tm; the resulting 
function is added to the set of basis functions B. Finally, the basis B is orthonormal- 
ized. Note that the orthonormalization is numerically challenging, as the basis func- 
tions are nearly linear dependent when span{B} is already a good approximation of 
the range of T; in [21] using the numerically stable Gram-Schmidt with adaptive reit- 
eration from [19] is suggested. The main loop of the algorithm is terminated when the 
following a posteriori norm estimator is smaller than the desired tolerance tol. 
Proposition 6.14 (A probabilistic a posteriori norm estimator [21]). Letr;,i=1,...,m, 
ben, random normal test vectors and fee and Ms. the smallest and largest eigenvalues 
of the matrix of the inner product in S. Then, the a posteriori norm estimator 


-1 
Alne, Etestfail) = Cest (Np Etestfail) mar I(T = P grana T) D; rill (6.46) 
sell 
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satisfies 


P{I|T — PpranaT || < ACM, Etestfai1)} = (1 - Etestfait)» (6.47) 


Ms 
min 


A(n, Etestfail) 

|e < Cop¢(Mp, Etestfail) ¢ Z 1- Etestfail» 
|T — PpranaT || 

n 


where Cost(Np Etestfail) = 1/[(2A )Y? erf 1 ¥/Etestfail) |. Additionally, we have 


where the constant Core(N}, Etestfai)) is defined as 


1/2 


/Nr € a \a™s 
1f Nr fail 1 
CeffNp Etestfail) = fo ( >? festa ) “Ms (erf ( /Etesttait)) 


Mt min 
and Q™ is the inverse of the upper normalized incomplete gamma function. 


The constant Cest(Ne Etestfai]) is calculated on line 3 using Ny, which denotes the 
rank of operator T. In practice Ny is unknown and an upper bound for Ny such as 
min(Ns, Ng) can be used instead. Note that the term (max;ey IItIlz) © Cest(Nt Etestfail) iS 
the norm estimator (6.46). The test vectors are reused for all iterations. 

To finally analyze the failure probability of Algorithm 6.1 we first note that after Ny 
steps we have R'™@"4 = range(T) and thus |T — PparaT|| = 0, yielding the termination 
of Algorithm 6.1. Using the fact that thea posteriori error estimator defined in (6.46) is 
therefore executed at most N; times combined with the probability for one estimate 
to fail in (6.47) and a union bound argument we infer that the failure probability for 
the whole algorithm is E,jgofair < NT Etestfail- 

Remarkably, the convergence behavior of the reduced space is only slightly 
worse than the rate A,,,, which is achieved by the optimal local approximation 
spaces defined in Theorem 6.11. 


rand 
Ra 


Proposition 6.15 (A priori error bound [21]). Let AMR and ANR denote the largest and 


smallest eigenvalues of the inner product matrix Mg and let R®™? be the outcome of 
Algorithm 6.1. Then, for n > 4 we have 


1 
. k r— en 2 
EIT ~ Pprana T| < CRs TE (2 te Vie i (D4) | (6.48) 


k>2,p>2 Bk 


M M 
where Crs = (Ami Ane) Uma 


It can be observed in numerical experiments that the a priori bound in Proposi- 
tion 6.15 is sharp in terms of the predicted convergence behavior as we will show now 
for a test case from [21]. Moreover, we will investigate the performance of Algorithm 6.1 
also for a test case from [21]. To that end, let On = (-0.5, 0.5) x (—0.25, 0.25) x (—0.5, 0.5) 
and Qm = (-0.5, 0.5) x (—0.5, 0.5) x (—0.5, 0.5) be the subdomains on which we aim 


270 —— A. Buhret al. 


to construct a local approximation space, Oout = (-2,2) x (-0.25, 0.25) x (-2,2) and 
Qout = (-2,2) x (-0.5,0.5) x (-2,2) the corresponding oversampling domains, and 
Tout = {-2,2} x (—0.25, 0.25) x (=2, 2) U (-2, 2) x (-0.25, 0.25) x {-2, 2} and Tout = {-2, 2} x 
(-0.5, 0.5) x (—2, 2) u (—2, 2) x (—0.5, 0.5) x {-2, 2} the respective outer boundaries. On 
00 su \ Pe and ðQout \Tout we prescribe homogeneous Neumann boundary conditions 
and we suppose that Qout and Q ut do not border the Dirichlet boundary of Q. For the 
FE discretization we use a regular mesh with hexahedral elements and a mesh size 
h = 0.1 in each space direction and a corresponding conforming FE space with linear 
FE resulting in dim(V;,l5, p) = 30,258, dim(R) = Np = 2,172, dim(S) = Ns = 2,880 for 
On, and Valo „ = 55,473, Nr = 3,987, and Ns = 5,280 for Oout- We equip the source 
space S with the L?-inner product and the range space R with the energy inner prod- 
uct. Finally, for all results in this subsection we computed the statistics over 1,000 
samples. 

Analyzing the convergence behavior of E(||T ~P piana T||) on Oout for a growing num- 


ber of randomly generated basis functions k and a (fixed) oversampling parameter p = 
2in Figure 6.4a we see that until k = 75 the a priori bound reproduces the convergence 
behavior of E(||T - Parana T||) perfectly. We may thus conclude that the a priori bound 
in (6.48) seems to be sharp regarding the convergence behavior of E(||T - Parana T|) in 
the basis size k. We also observe that the a priori bound is rather pessimistic as it over- 
estimates E(||T -Prena T||) by a factor of more than 100; this is mainly due to the square 
root of the conditions of the inner product matrices. 

Regarding the performance of Algorithm 6.1 on Q,,, we first observe in Figure 6.4b 
that the actual error ||T — Pprana T|| lies below the target tolerance tol for all 1,000 sam- 
ples for n; = 10, which holds also true for all other considered values of n;. Here, we 
prescribe €ajgofail = 107° and use 3,993 as an upper bound for Ny. We see in Figure 6.4b 
that increasing the number of test vectors n, from 5 to 10 or from 10 to 20 increases the 
ratio between the median of the actual error ||T — PpranaT|| and the target accuracy tol 
significantly — for the former by more than one magnitude — while an increase from 
n, = 40 to n; = 80 has hardly any influence; similar results have been obtained in [21] 
for heat conduction and a Helmholtz problem. This can be explained by the scaling 
of the effectivity of the employed a posteriori error estimator, which is of the order of 
1,000 for n, = 5 and of the order of 10 for n, > 20. Regarding the choice of n, it seems 
that for the present test case a value of about 20 is in the sweet spot. We thus infer 
that for the present test case only very few local solutions in addition to the optimal 
amount are required, demonstrating that Algorithm 6.1 performs nearly optimally in 
terms of computational complexity for the current problem. 


3 Note that although in theory we should subtract the orthogonal projection on the six rigid body 
motions from the FE basis functions, in actual practice we avoid that by subtracting the orthogonal 
projection from the harmonic extensions only. 
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Figure 6.4: (a) Comparison of the convergence behavior of \/Ax,.1, Yhd E(IIT — Pgk+p TIl), the a 

priori error bound (6.48), and the a priori error bound of (6.48) scaled with a constant such that its 

value for k = 2 equals the one of E(||T — Pprana TII) (sc. a priori) for increasing k for and p = 2 for the 
k+p 


oversampling domain Qout: (b) Median of the projection error ||T — P rand T|| for a decreasing target 
accuracy tol for a varying number of test vectors n; and the minimal and maximal values for n; = 10 
on Oout- 


6.4.2.3 The general setting P + {pi} 


The processes in Sections 6.4.2.1 and 6.4.2.2 yield for every u € P the local approxi- 
mation space R; (u) for this specific parameter u € P; R} (u) can also be generated by 
some other process, where we require that 


E 


= 64 
2C1(Tp H) om 


[TO - Pr TQ) < 
possibly only at high probability and that Rý (u) is defined as the direct sum of R„(4), 
the kernel of the bilinear form, and representations of nonhomogeneous Dirichlet 
boundary conditions and the right-hand side. We abuse notation in this subsection 
by omitting henceforth the remark that the estimate may only hold in a probabilistic 
sense. The constant C,(7;,, H) has to be chosen in such a manner that if one uses the 
parameter-dependent spaces R} (y) to define u” (u), we have 
luga) — un, Il 2 


<x. 6.50 
Iu) Ie 2 on 


The spectral greedy algorithm as introduced in [103]* constructs one (quasi- 
optimal) parameter-independent approximation space Ry which approximates those 


4 For a generalization to a setting where the discrete parameter set describes different geometries 
such as a beam with or without a crack we refer to [101]. 
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parameter-dependent spaces R} (u) with a given accuracy on a finite-dimensional 
training set = c P. In the spectral greedy algorithm we exploit the fact that we expect 
that the local spaces R} (u), and in particular the spectral modes that correspond to 
the largest eigenvalues, are not affected too much by a variation in the parameter 
thanks to the expected very rapid decay of the higher eigenfunctions in the interior 
of Qout- 

The spectral greedy as described in Algorithm 6.2 then proceeds as follows. Af- 
ter the initialization we compute for all u € E the parameter-dependent spaces R} (p) 
such that we have (6.49). Note that for a decomposition Ty with mutually disjoint in- 
terfaces (also called ports), where each Qm, m = 1,...,M, has exactly two interfaces 
and all interfaces have the same geometry, we have the following a priori error bound 
[103] for the error between u(y) and the continuous port-reduced static condensation 
approximation u, (M) corresponding to the parameter-dependent optimal interface 
space R} (p): 


luga) — un, Il 


ool, * #yC\(H)Co(H) ma (Cala Ayn). (6.51) 


Here, the constant C, (Q,, 4) depends only on the subdomains that share y and not on 
Q or on u(y). Moreover, c, (p) and c,(p) are chosen such that we have c,()lll-llly < I-l < 
C2(H)Il-lilp for all u € P and a fixed reference parameter ñ € P. Choosing Ci(Ty, M) = 


Algorithm 6.2: Spectral greedy [103]. 
Input : train sample E€ c P, tolerance € 
Output: set of chosen parameters Ey, local approximation space Ry 


1 Initialize N — dim(ker(a,,(-,v))), 
Ey — Ø, Ry — ker(am(» v)) or Ry — ker@n(.v))Ir,_, 
2 foreach p € Edo 


3 Compute R, (p) such that |T (p) - Preg) TDI < LOH 


4 while true do 
5 | ifmaxycs E(S(R,(H)), Ry) < €/(€ + 2Cx(Ty p)c(p)c(p)) then 
6 p return 


7 | př = arg maxņes E(S(R} (4) Rw) 


8 Ena © Ey UM" 
9 | K argSuppesir: q) fre, IP - Cllr 
10 Ry+1 — Ry + Span{x} 


11 NeN+1 


12 return Ey, Ry 
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#yc,(W)C>(M) maxX ery Cy (Q, 4) and un 41(H) < €/2yields a reduced space R} (u) that 
satisfies the requirements stated in the beginning for every u € ©. Although precise 
estimates for Cy ;(O,, 4) can be obtained, setting C, ;(Q,,M) = 1 yields in general good 
results as another value would just result in rescaling £; for further details see [103]. 
After having collected all functions on Tm , or Qn that are essential to obtain a good 
approximation for all local solutions u),,(H) of the PDE evaluated on Ty w OF Om H € E, 
we must select a suitable basis from those functions. This is realized in an iterative 
manner on lines 5-14. 

In each iteration we first identify on line 7 the reduced space R} (u*) that mazi- 


mizes the deviation 


E(S(R,(H)),Rw):= sup inf I -¢lp} MES 
EeS(Rt (p)) ERN 


where possible choices of S(R} (9)) c R} (y) will be discussed below. Subsequently, we 
determine on line 9 the function x € S(R}(p*)) that is worst approximated by the space 
Ry and enhance Ry with the span of x. The spectral greedy algorithm terminates if for 
all u € E we have 


max E (S(Rn QD)» Ry) < €/(E + 2C2(Tip We (H)C2()) (6.52) 
for a constant C (Ty, 4), which can in general be chosen equal to one. We emphasize 
that both C,(7;;,) and C,(7;,, H) do in general only depend on the number of faces or 
subspaces on which the respective reduced space Ry is used and not on the precise de- 
composition of Q; see (6.51). A slight modification of the stopping criterion (6.52) and 
a different scaling of £ in the threshold for the a priori error bound on line 3 allows to 
prove that after termination of the spectral greedy for a decomposition 74 with mutu- 
ally disjoint interfaces, where each Qm, m = 1,...,M, has exactly two interfaces and 
all interfaces have the same geometry, we have [103] 


luga) — un W Uh / Tle ely < E- (6.53) 


Here, uy(p) is the continuous port-reduced static condensation approximation corre- 
sponding to Ry, Ry being the continuous outcome of the spectral greedy. 


Choice of the subset S(R* (u)) 

First, we emphasize that in contrast to the standard greedy as introduced in [111] 
we have an ordering of the basis functions in Rt (u) in terms of their approximation 
properties thanks to the transfer eigenvalue problem; the sorting of the basis func- 
tions in terms of their approximation properties is implicitly saved in their norms as 
Ix; (wile = A(H),j = 1,...,n. To obtain local approximation spaces Ry that yield a 
(very) good approximation w (p) already for moderate N it is therefore desirable that 
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the spectral greedy algorithm selects the lower eigenmodes sooner rather than later 
during the while loop. As suggested in [103] we thus propose to consider 


S(R 0D) = {ÇZ € RW : Ilr < Uh (6.54) 
1/2 


with [Odlar = (Sew) | 
i=1 


where ((m) = Y3",¢;@0xi(#), n, = dim(R* (qu), and here and henceforth {y;()};, 
denotes the orthonormal basis of R} (u). Note that we are therefore considering a 
weighted norm in R} (u). The deviation E (S(R} (H)); Ry) can then be computed by 
solving the following eigenvalue problem: Find (0;(H); 0;(H)) e (IR™, R*) such that 


Z(W)e;(H) = 9;)e;(), 


k=1 k=1 


N N 
where Z;)(H) := (109 - È CDX) Xo Xi) - S ogona) 
R 


and xx denotes the orthonormal basis of Ry. We thus obtain E(S(R}(4)), Ry) = Vo,(M), 
for all u € E, and k = ee 0,(H" )y;(H") at each iteration. 

Note that were we to consider the norm || - ||p in (6.54) the sorting of the spectral 
basis x;(j4) of R} (u) in terms of approximation properties is neglected in the while loop 
of Algorithm 6.2; for further explanations see [103]. 

Finally, we compare in Figure 6.5 the spectral modes generated by the spectral 
greedy algorithm, Algorithm 6.2, numerically with other interface modes, demonstrat- 
ing the superior convergence of the former. In detail, we compare the relative error of 
the port-reduced static condensation approximation for interface spaces comprising 
“Legendre polynomial”-type functions? [37], empirical port modes constructed by a 
pairwise training algorithm® [37, 38], and the spectral modes. To that end, we consider 
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O O 
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Figure 6.5: |u, (4) — u" (W) y/u Gy for the 
30 Legendre, empirical, and spectral interface 
N basis functions for the solid beam. 
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5 Note that each component of the displacement is the solution of a scalar singular Sturm-Liouville 
eigenproblem. 

6 Following the notation in [38] we have chosen Ngampies = 500 and y = 3 in the pairwise training 
algorithm. 
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a domain Q which consists of two identical solid beams, each of whom is associated 
with a subdomain Q;, i = 1, 2. Here, we choose Q, = (-0.5, 0.5) x (—0.5, 0.5) x (0,5), Q, = 
(-0.5, 0.5) x (—0.5, 0.5) x (5,10), and Tout = [T1 U2, with T; = (—0.5, 0.5) x (—0.5, 0.5) x {0} 
and T, = (-0.5, 0.5) x (—0.5, 0.5) x {10}. The underlying FE discretization has N = 3,348 
degrees of freedom per subdomain and Np = 108 degrees of freedom per interface. We 
require E(u) to be uniform within each subdomain, the constant varying in [1,10], and 
choose for G() € R? the admissible set of parameters to be [-1,1] x [-1,1] x [-1, 1]. 
Finally, we equip both S and R again with a lifting inner product. Within the spectral 
greedy we have considered 200 parameter values sampled from the uniform distribu- 
tion over P and £ = 1-10°°. On average the interface spaces R} (p) have a size of 13.65 
and the resulting parameter-independent port space Ry has a size of 56. 

In the online stage we consider E(u) = 1 in both components, G = (0,0, 0)", and 
prescribe gp; = (0,0, 0)? at T; and $p2 = (1,1, 1)’ at T». We observe that the Legendre 
modes perform by far the worst, demonstrating that including information on the so- 
lution manifold in the basis construction procedure can significantly improve the ap- 
proximation behavior. We remark that the Legendre modes will perform even worse in 
the case of less regular behavior on the interface, which further justifies the need for 
problem-specific local approximation spaces in the sense of model reduction. The em- 
pirical modes and spectral modes exhibit a comparable convergence until N = 17, but 
for N > 17 the relative error in the spectral approximation is one order of magnitude 
smaller than that of the empirical port mode approximation. This can be explained by 
the fact that thanks to its conception the pairwise training algorithm is able to iden- 
tify and include the most significant modes, but (in contrast to the spectral greedy 
algorithm) might have difficulties to detect subtle modes that affect the shape of the 
function at the interface Tm w only slightly. Note that the temporary stagnation of the 
relative error for N = 7,...,17 for the spectral modes is due to the fact that the spectral 
greedy prepares the interface space for all possible boundary conditions and param- 
eter configurations. Thus, for the boundary conditions considered here some spectral 
modes, as, say, a mode related to a twisting (torsion) of the beam, are not needed for 
the approximation. 


6.5 A posteriori error estimation 


6.5.1 Residual-based a posteriori error estimation 


A global residual-based a posteriori error estimator for projection-based model reduc- 
tion is readily defined as 


1 
A(uy(H)) = —— | Run w; lly (6.55) 
a(H) h 
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where R(uy(H);H) € V, is the global residual given as (R(uy (H); U); Pn) = fa(Pni) - 
Ay (Uy (H); Py; H) for all p, € Vp. This error estimator is known to be robust and efficient 
(cf. [50, Proposition 4.4]), i.e., we have 


lun) - un Gly < Ady GD) < ae E lungo - un QD (6.56) 


For localized model order reduction, ee we are merely interested in local- 
ized a posteriori error estimation. To this end, we first present abstract localized lower 
and upper bounds for the dual norm of a linear functional (see [20]). 


Theorem 6.16 (Localized lower and upper bounds for functionals). Let O;,1 < i < M, 
be a collection of linear subspaces of Vp, and let Po, : Vn — O; S Vp be mappings 
which satisfy pen Po, = idy,. Moreover, assume that for J € N there exists a partition 
Ware = {1,...,M} such that for arbitrary 1 < j < J andi, + i, € Y; we have O; L O;,. 
Defining the stability constant of this partition modulo Vy as 
M. -Iai 
(S74 infpev,no, IPo,(@) - GIP)? 


Cy := sup —_ ; (6.57) 
NT pevao) loll 


we have for any linear functional f € V, with (f, p) = O Yọ € Vy the estimate 


(Svg ) ew <c (Er) (6.58) 
VJ = O; > Vv), +N L o! . 


Here, |if ||g denotes the norm of the restriction of f to O;. 


When grouping the spaces O; so that in each group, all spaces are orthogonal to 
each other, J is the number of groups needed. Note that subtracting the projection onto 
Vy in (6.57) allows subtracting, say, the mean value of a function or the orthogonal 
projection onto the rigid body motions, if the respective functions are included in Vy. 
We may thus employ, say, Poincaré’s inequality or Korn’s inequality in subdomains 
that do not lie at Tp. 

Applying both estimates to the residual R(uy (4); 4) € V}, we obtain from (6.55) 
and Theorem 6.16 a robust and efficient, localized error estimate. 


Corollary 6.17 (Localized residual-based a posteriori error estimate). Let the assump- 
tions on the subspace collection O; and the mappings P; from Theorem 6.16 be satisfied. 
Then, the error estimator A\,.(Uy(H)) defined as 


i 
2 


Mtoc(Un )) := age > [RC Uy (H); 4 Plo, (6.59) 


is robust and efficient, i. e., 


yu) Vey 


ao lungo) — Uy G2) | (6.60) 


Jurgo) z un(Wlly = Moc(Un(H)) = 
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Online-offline decomposition of this error estimator can be done by applying the 
usual strategy for online-offline decomposition used with the standard reduced basis 
error estimator (see, e. g., [50, Section 4.2.5] or a numerically more stable approach 
[19, 25, 105]) to every dual norm in A,,.(uy()). 

The a posteriori error estimator for the ArbiLoMod derived in [20] and the a poste- 
riori error estimator for the scRBE method as suggested in [100] both fit into the frame- 
work above, as will be detailed below in Examples 6.18 and 6.19. In contrast, for in- 
stance the error estimators proposed in [61, 60] for the scRBE method exploit matrix 
perturbation analysis at the system level to bound the Euclidean norm of the error be- 
tween the coefficients of the static condensation solution and the coefficients of the 
static condensation solution using a reduced basis approximation in the interior. To 
estimate the error caused by interface reduction in [37] a computationally tractable 
nonconforming approximation to the exact error is employed. To take into account 
the error due to the intra-element reduced basis approximations ideas from [61] are 
used. It can also be noted that the error estimators in [61, 60, 37] are only valid under 
certain assumptions on the accuracy of the reduced basis approximation. In [83] a lo- 
calized a posteriori error estimator for interface reduction and intra-element reduced 
basis approximation is presented for the coupled Stokes—Darcy system. The a poste- 
riori error estimator for the CMS method derived in [64] employs the dual norms of 
residuals and eigenvalues of the eigenproblems used for the construction of the (lo- 
cal) basis functions. The error estimator in [64] is however only partially local as it 
involves the residual for the port or interface space on the whole interface T. For lo- 
calized a posteriori error estimation in the context of adaptive GMsFEM we refer to 
[31, 30, 28, 29]. 


Example 6.18 (Localized a posteriori error estimate for ArbiLoMod [20]). Let us as- 
sume V, c V = HQ) and choose O; as subspaces of H'(Q;), where Ty := {Q,,..., Ou! 
is an arbitrary overlapping decomposition of Q, which may be chosen independently 
from Ty. Assume that there is a partition of unity p; € H>@(Q;)n C(Q;), X4 pi = 1, 
such that |p; < land ||Vpil., < Cpu diam(Q;)!. The constant Cpu Will depend on 
the size of the overlap of the subdomains Q; with their neighbors in relation to their 
diameters. 

Moreover, we assume that there is a linear interpolation operator Z onto Vp such 
that Z is the identity on V, with Z(p;V,) S O; and ||Z(pivn) — PiVally < CrllVallg, ; for all 
Vh € Vp. We then can define mappings i 


Po, (Vp) = TOPi ` Vh), 


which satisfy the assumptions of Theorem 6.16. In case V, comes from an FE discretiza- 
tion, a possible choice for Z is Lagrange interpolation. 

If we now ensure that the partition of unity p; is included in Vy, we can choose @ in 
the definition of cy as @ := p;-|Q;|"* Ío, gy, which allows us to prove [20, Proposition 5.7] 
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that cy can be bounded by 


Cy < Jı + 2C? + 4(CpuCpe)? * VCovip: 


In this estimate Coyjp := MaXyeg #{i € Yg |x € Q;} is the maximum number of estimator 
domains Q; overlapping in any point x of Q, and cpc is a Poincaré inequality constant 
associated with Te In particular, this result shows that the efficiency of (6.59) is inde- 
pendent of the number of subdomains in 7p, provided that the partition of unity p; is 
included in Vy. 


Example 6.19 (ScRBE method and interface reduction from [100]). We exemplify the 
a posteriori error estimator from Corollary 6.17 for the scRBE method, which is 
equally applicable when considering solely static condensation and no intra-element 
reduced basis approximations.’ To simplify notations we define interface spaces 
Vi := span{py,..., ie where NY = dim(V;l,); for the definition of YY we refer 
to Section 6.3.1. Recall ‘that we then have the following space decomposition of the 
(global) FE space V: 


M 
Via = DV ino ® ( py | (6.61) 
m=1 


yeTy 


We may thus uniquely rewrite every p € V, as 
M 
p=} o"+ Vg, (6.62) 


where y™ € Vj", and o” € VÝ, extending y™ and g” by zero. This allows us to define 
mappings Pym : Vp > Vio P > ”™ and Py : Vp > VÝ, p = 9”, as required in 
Theorem 6.16. Thanks to (6.21) we also obtain 


I 
Vro L Vro m+m and Vio VY, m=1,...,Q,y € T}. 


It thus remains to verify that we can bound the constant cy with Vy as defined 
in (6.28). To that end, we first note that thanks to (6.21) we have the following stability 
result [100, Proposition 4.1]: 


2 


y 
Lv 


yeTy 


M 
lol = dle" + (6.63) 
m=1 


7 The error estimator in [100] is derived for mutually disjoint interfaces. However, we conjecture that 
the estimator can be generalized to general decompositions of Q. 
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We thus obtain 
M1 lOmlly + Eper inf, evy 1O - eI)? 
È YEl H Prev 
Cy < sup 

peV;\{0} lolly 
(6.63) dely + Eyer infpy eyy IP” - IG)? 
< sup > 

peV;,\{0} lolly 


where V}, := span{y},..., YXy}. To show Lyer infgrev? lp” - 1È, < cllglly for a con- 
stant c we choose @” such that (@” — Ply equals the trace of @ minus the orthogonal 
projection on the kernel of the bilinear form; for further details see [21, 103]. Then, we 
can use [103, Lemma B.4] to conclude boundedness of cy and thus (6.60), the latter 
corresponding to [100, Proposition 4.2 and Corollary 4.6]. 

Finally, we shortly discuss how to compute the dual norms of the residuals 
in (6.58). The dual norms of the residuals of the intra-element reduced basis ap- 
proximations can be computed by employing Riesz representations (see for instance 
Chapters 1 and 4 of this volume of Model order reduction). The dual norms of the 
residuals in the interface space can be computed by means of conservative fluxes [57], 
which have been extended to interface reduction in [100]. In detail, we compute the 
conservative flux Hy (4) such that 


> (AN) B")y = fn WH) -amun py) vp « GB Vy, (6.64) 


YEQnm YEQm 


where (., )y denotes a suitable inner product on the interface y. Note that thanks to our 
mutual disjoint interface assumption problem (6.64) decouples and we may compute 
the conservative flux separately for each interface y. Moreover, by orthonormalizing 
the interface basis functions Xk defined in Section 6.3.1 with respect to the (-,-), inner 
product, the computation of H (u) reduces to the assembling of the residual in (6.64). 
The computational costs thus scale linearly in (Ny —NY) and N”. For further details we 
refer to [100]. 


6.5.2 Local flux reconstruction-based error estimation 


Following [90], we discuss local flux reconstruction-based a posteriori error esti- 
mation of the full approximation error u(u) — uy() (that is, the discretization as 
well as the model reduction error) in the context of nonconforming approximations 
of elliptic multiscale problems such as Example 6.2. An extension to convection- 
diffusion—-reaction problems based on [39] is straightforward. This estimate was in- 
troduced in the IP localized nonconforming setting of the LRBMS method (compare 
Section 6.3.2.2). 

Recalling the broken Sobolev space and broken gradient operator from Sec- 
tion 6.3.2.2, the key idea of flux reconstruction-based error estimation is to observe 
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that not only the approximate solution u,(j) is nonconforming, but also the approxi- 
mate diffusive flux —x(j)V,Uy(), in the sense that it is not contained in H;,(Q) (i. e., 
the space of functions in L?(Q)? whose divergence exists in a weak sense and lies in 
L’(Q)). 

We may then obtain computable estimates by comparing these quantities with 
conforming reconstructions, as detailed further below.® The respective reconstructed 
diffusive flux is locally conservative and is related to the conservative flux reconstruc- 
tion to compute the dual norm of the residuals in the interface space in Example 6.19. 

To begin with, we specify the parameter-dependent (semi-)energy norm induced 
by the bilinear form a for a parameter pt € P, [l|-llp : H! (Th) > R, v > livlig == alv, vj)? 
(by using the broken gradient in the definition of a) and note that we can compare 
these semi-norms for two parameters by means of the affine decomposition of a (com- 
pare (6.69)), 


1/2 ere avi 
livlig < lvl, < 84 A)? vl 


Oa Ä) 
with the equivalence constants given by ©,(} ñ) := mines, 01u) O40) and 
0,4 = maxes, ©4(u) omt, respectively. The first abstract result is the following 
discretization-agnostic lemma, which leaves the choice of the reconstructions, v and 
s, open. (We give estimates on the full V;,-norm at the end of this subsection.) 


Lemma 6.20 (Abstract energy norm estimate (Lemma 4.1 in [90])). For y € P, let 
u(u) € V denote the weak solution of (6.1) with the data functions x and q as in Ex 
ample 6.2. Then for arbitrary vy € H'(t),) and jt € P, we have 


— 1 
lugo) -vylla < 9a AY? {O14 A)? inf Nu — vil 


an a eee O TS voeo} 
lPlap=1 

1 

2 


e 
HP hug) -vyl 


To obtain a fully computable localizable estimate we need to specify the conform- 
ing reconstruction of the solution (v in the above lemma) and of the diffusive flux (s in 
the above lemma). We define both reconstructions with respect to the global fine grid 
T, and note that their respective computations can be localized with respect to the 
domain decomposition to allow for offline-online decomposable localized estimates. 


8 Note that the entire analysis holds for the FOM solution u;,(j) as well as the ROM solution uy (jp) 
(compare [90]), but we restrict the exposition to the latter. In particular, the presented estimates can 
thus also be used to steer grid adaptation of the FOM solution. 
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We reconstruct the nonconforming solution u,(u) € Vp by means of its Oswald in- 
terpolant Ioslu,(u)] € V. We define the corresponding Oswald interpolation operator 
Ios : Vp > Vp N V by specifying its values on each Lagrange node v of Tp: Given any 
Vh € Vh, we set Ips[Vp](V) := Vale(v) for any Lagrange node lying inside a grid element 
t€Tp, 


Ioslva](v):=0 for all boundary nodes and Ios[va](v) := a >, Vale) 
h 


V 
tet) 


for all nodes which are shared by multiple grid elements, which we collect in T% ¢ Tp. 
The definition of the conforming reconstruction of the nonconforming diffusive 

flux -xK(W)V,U,(H) € L?(Q)4 is more involved. Given l > 0, we define the I-th-order 

Raviart-Thomas—Nédélec space of vector-valued functions by 


RIN} (Tp) := {s € Hyiy(Q) | sle € [P(E] + xP (t) Vt € T,} 


and note that the degrees of freedom of any Sp € RTN! (Tp) are uniquely defined by 
specifying the moments of order up to l — 1 of s,|, on all elements t € T, and the mo- 
ments of order up to l of s,|, -n, on all faces o € T) (compare [18]). With these prelim- 
inaries we define the diffusive flux reconstruction operator R! : P > [Vp > RIN;,(T;)], 
given some v, € V, and some ų € P by specifying the degrees of freedom of R} [Vn H] € 
RTN} (Th), such that 


(R! Ivy; p] - Nos) 12.9) = 4 Vw rH) + Vpr), forallr ¢ Po) (6.65) 


onallo € T% and 


(Rilvns H} Vr) = -4° (Rrlvsulornu)- X apt vwm (6.66) 


y 
OET, Nt 


for all Vr <€ [P,_,(t)]4 with r € P(t) on all t € T}. Given a FOM space V, of polyno- 
mial order k > 1, we choose a (k — 1)-th order reconstruction. With this definition, 
the reconstructed diffusive flux of a given reduced solution u,(y) fulfills the following 
local conservation property, given that the constant function 1 is present in the local 
reduced spaces Vý: 


(V -RK [uv WsHl D2, = (qD, forall Om € Ty- 


When inserting this diffusive flux reconstruction for s in Lemma 6.20, this local con- 
servation property is key to obtaining the following estimate. 


Theorem 6.21 (Locally computable energy norm a posteriori estimate). Let the do- 
main decomposition Ty from Definition 6.5 be such that the Poincaré inequality holds 
on each subdomain Q„ € Ty with a constant CẸ > O, 


2 
lo - Toela, < CP hm Volio, forall € H'(Q,), 


282 — A. Buhretal. 


where h,, := diam(Q,,) and where II5'g denotes the mean value of p over Qm. Let further 
u(u) € V be the weak solution of (6.2) and let uy(u) € Vy be the IP localized ROM 
solution, with 1 € Vý for 1 < m < M. Then for arbitrary ja, jt € P, we have 


lu) - Uy lla < nC; Bw) 


with the a posteriori error estimator n(H; ji; fi) given by 


nasi) = Comp an F neoa) 
On €T 


(E aban +e" naa) ], 


Om€TH 


and the local nonconformity, residual, and diffusive flux indicators given by 


Tht (Ms BD) == \(Vv) - Los [Vv(H))lo, las 


Co, : 
ne" (Ww) = E lla-V-R, ‘Luv ®s# lee,» and 


Nar BD) := |e) *(«() Vjtew (H) + Rh [uv )sH)Il2~0,> (6.67) 
respectively, where Ko denotes the minimum eigenvalue of x over Q and P. 


We obtain an a posteriori error estimate with respect to the V -norm or a full en- 
1 
ergy norm, |l-[l, + (Zoer (> -)2)2, by noting that (u(y), u())? = O for a weak solution 
u(p) of sufficient regularity. 


6.6 Basis enrichment and online adaptivity 


Model order reduction is usually employed either (i) in the context of real-time deci- 
sion making and embedded devices or (ii) in the context of outer loop applications, 
such as optimal control, inverse problems, or Monte Carlo methods. In (i), one is 
usually interested in reduced spaces Vy of very low dimension to obtain ROMs as 
small as possible, at the possible expense of very involved offline computations. 
Here, localized model order reduction may help to reduce the latter, but we can 
usually not expect the resulting reduced space to be smaller than the one gener- 
ated using traditional global model order reduction methods. In (ii), however, one 
is interested in a black-box-like approximation scheme which is queried for a huge 
amount of parameters, with a somehow “optimal” computational cost (including of- 
fline as well as online cost). Here, one may keep high-dimensional data throughout 
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the computational process (offline as well as online), and it is in this context that 
localized model order reduction techniques may truly outperform other approaches. 
In the context of PDE-constrained optimization this has been investigated, e. g., in 
[91, 88, 116]. 

The localized a posteriori error estimation as discussed in Section 6.5 enables 
adaptive enrichment of the local reduced approximation spaces, whenever the quality 
of the reduced scheme is estimated to be insufficient — be it due to insufficient training 
due to lacking computational resources or due to limited knowledge about the range 
of possible parameters or due to other reasons altogether. 

Let us thus assume that an initial (possibly empty) localized reduced approxima- 
tion space Vy is given, compare Section 6.4. The goal of an adaptive enrichment is 
to enlarge the local solution spaces with additional modes that reflect nonlocal influ- 
ences of the true solution such as channeling effects or singularities. Local adaptive 
basis enrichment can be employed both offline for the whole parameter range and/or 
online for a specific chosen parameter. Empirical training followed by offline enrich- 
ment is, e. g., used in a greedy manner for the basis construction in ArbiLoMod (cf. 
Example 6.18) in [20]. Adaptive enrichment for the GMsFEM is presented in [31, 30] 
and online adaptive enrichment in [28, 29]. For the exposition in this section, we re- 
strict ourselves to online enrichment as introduced in [90], i. e., for local enrichment 
of the basis when a certain parameter is already chosen. 

From a bird’s-eye perspective, we can think of an online adaptive reduced scheme 
as a p-adaptive FE scheme with problem-adapted basis functions, where the local re- 
duced bases are adapted during online enrichment.’ Thus, we can think of online en- 
richment in the usual Solve — Estimate — Mark — Refine (SEMR) manner, well known 
in grid-adaptive discretization schemes. In the Estimate step we employ an a posteri- 
ori error estimate y that is localizable with respect to the domain decomposition, i. e., 
n < SXi Nn with appropriate local indicators nm. Examples are given in Section 6.5. 
As such, most marking strategies from grid-adaptive schemes are applicable, and we 
give examples in Section 6.8.1. In this context, refinement is locally done by enrich- 
ing the local reduced spaces, that is, by adding additional basis functions to the local 
reduced bases on selected subdomains. We thus presume we are given a parameter 
H € P and a reduced solution uy(w) € Vy, the estimated error of which is above a 
given tolerance. 

As an example, we detail the online enrichment procedure used in the context 
of the LRBMS (compare Section 6.3.2.2), using the a posteriori error estimation tech- 
niques from Section 6.5.2. Inspired by domain decomposition as well as numerical 
multiscale methods, we may then obtain a candidate for the next element of a local 


9 We would also like to mention the h-adaptive model order reduction approach from [24], which 
is based on a k-means clustering of the degrees of freedom, but we restrict the exposition here to 
localization with respect to a domain decomposition. 
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reduced basis by solving local corrector problems on a collection T4 € Ty of marked 
subdomains with u,(H) as boundary values. For each marked subdomain Qm € Ty, 
we denote by Om := {Qm € TH | Qn A Qw + O} an overlapping subdomain and by 


V M fla, lve Viv Va0, = 0} the associated restricted FOM space, encoding zero 


Dirichlet boundary values. We are then looking for a local correction go € ve m, such 
that 


an( 9°", vis H) = fars H) - an(Un(H)la p Yn) forallvy € a (6.68) 


where we understand all quantities to be implicitly extended to Q by zero, if required, 
and note that p°" can be computed involving only quantities associated with Q,,. Us- 
ing this local correction on the overlapping subdomain, we obtain the next element 
of the local reduced basis associated with Q,,, by an orthonormalization of (p° + 
Uy(H))|o,, With respect to the existing basis on Vy 

Given a marking strategy and an orthonormalization procedure, we summarize 
the adaptive online enrichment used in the context of the LRBMS in Algorithm 6.3. 


Algorithm 6.3: Adaptive online enrichment in the context of the LRBMS. 
Input :a marking strategy MARK, an orthonormalization procedure ONB, a 
localizable offline-online decomposable a posteriori error estimate 
nw)? < Y% Nm’, local reduced bases O” for 1 < m < M, p € P, 
uy (p), Aonline > 9 
Output: Updated reduced solution 
10" — o", yi<m<M 
2n<-0 
3 while n(p) > Aontine do 
4 forall 1 < m < M do 
5 compute local error indicator nm(Ņ4) 


6 TH = MARK (71, {1m(Mh<m<m) 
7 forall ©,,, € 7y do 


8 | Solve (6.68) for pn PD? — ons D", (9 + uy (W))Io,,}) 


9 update all reduced quantities (system matrices, error estimates) with respect 
to the newly added basis elements 
10 solve (6.13) for the reduced solution uy (jz) using the updated quantities 


1 return uy(H) 
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6.7 Computational complexity 


In this section we discuss the computational efficiency of localized model order re- 
duction schemes in comparison to standard, nonlocalized techniques. Imposing a lo- 
calization constraint on the reduced space naturally yields suboptimal spaces in the 
sense of Kolmogorov N-width. However, this is mitigated by the sparse structure of the 
resulting reduced system matrices. In particular, for problems with high-dimensional 
parameter domains with localized influence of each parameter component on the so- 
lution, we can expect localized ROMs to show comparable or even better online effi- 
ciency in comparison to a standard ROM. In addition, localized model order reduction 
provides more flexibility to balance computational and storage requirements between 
the offline and online phases and has thus the potential to be optimized with respect 
to the specific needs. This is particularly favorable for large-scale or multiscale prob- 
lems, where global snapshot computations are extremely costly or even prohibitive. 

In the offline (and enrichment) phase of the localized schemes, only relatively 
low-dimensional local problems are solved instead of the computation of global 
solution snapshots. In comparison to a global reduction approach with a parallel 
solver for snapshot generation (e. g., a domain decomposition scheme), the prepara- 
tion of the local reduced spaces via training (Section 6.4) can be performed almost 
communication-free, allowing the application of these schemes on parallel compute 
architecture without fast interconnect such as cloud environments. Via adaptive en- 
richment of the approximation spaces — based on the solution of local correction 
problems (Section 6.6) — smaller and more efficient ROMs can be obtained. In com- 
parison to domain-decomposition methods, where similar correction problems are 
solved, these correction problems are only solved in regions of the domain where the 
approximation space is insufficient. Thus, for problems with a localized effect of the 
parameterization, a significant reduction of the computational effort can be expected 
in the reduced basis generation process. 

In the context of component-based localized model order reduction (e. g., CMS, 
scRBE, reduced basis hybrid method, RDF) large computational savings can be 
achieved by the preparation of local approximation spaces (components) with re- 
spect to arbitrary neighboring components (connected through so-called ports). In 
addition to parametric changes of the governing equations or computational domain, 
this allows the (nonparametric) recombination of components in arbitrary new con- 
figurations without requiring additional offline computations. 


6.7.1 Online efficiency 


In view of Definition 6.8, we can interpret the localized model order reduction methods 
introduced in Section 6.3 as standard projection-based model reduction methods — 
such as the reduced basis method — subject to the constraint that the reduced space 
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Vy admits a localizing decomposition of the form (6.12). As such, the usual offline- 
online decomposition methodology can be applied. To this end, let us assume that the 
bilinear form a(., ; u) and the source functional f(-; y) admit affine decompositions 


Qa Q; 
a(v,w;p) = > O%(watvsw), fiw:m) = > OF fw), (6.69) 
q=1 q=1 


for all v, w € V, u € P with nonparametric bilinear forms af : V x V > R, functionals 
f1 € V', and some parameter functionals 04, of : P — R. If the given problem is not of 
the form (6.69), we can employ empirical interpolation [13] to compute an approximate 
affine decomposition. 

We begin by computing the reduced approximation space Vy using the methods 
outlined in Section 6.4. After that, a reduced model is assembled by computing matrix 
representations A? € R" of af and vector representations F? € R" of f1 with respect 
to a given basis 94, ..., Py Of Vy, i. €., 


AY := af (v; w;), F} := f4(w). (6.70) 


After this computationally demanding offline phase, the coordinate representation 
Up(H) € RY of the reduced solution Uy(H) of (6.13) is quickly obtained for arbitrary 
new parameters u by solving 


Qa Q 
>, OSA: Uy) = X Of ME? (6.71) 
q=1 q=1 


in the following online phase. The computational effort to determine uy (H) is of order 
O(Q,N? + QN) + O(N?) (6.72) 


for the assembly and solution of the dense equation system (6.71). In particular, we 
have obtained full offline-online splitting, i. e., the effort to obtain U,() is indepen- 
dent of dim V,. From Uy(y) we can then either reconstruct uy (H) by linear combina- 
tion with the reduced basis or evaluate arbitrary linear functionals of uy(y) by addi- 
tionally computing vector representations of these functionals in the offline phase. 

Reduced basis methods aim at constructing reduced spaces Vy which are near- 
optimal approximation spaces for the discrete solution manifold {u;,(y) |u € P} in the 
sense of Kolmogorov, i. e., we should have 


, (6.73) 


apr lur) -v|| = dy := h os inf Ju, -v| 
im W= 


where dy is the Kolmogorov N-width of the solution manifold. Localized reduced basis 
methods aim at reducing the computational effort of the offline phase by replacing 
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the computation of solution snapshots u;() of the global discrete full-order model by 
solutions of smaller localized problems associated with the domain Ty (see below). 
This comes at the expense of replacing the set of all N-dimensional subspaces of V, 
by the smaller set of all N-dimensional subspaces of V, of the form (6.12), i. e., we aim 
at constructing Vy with 


m a loc oS à a _ 
u PG) —vix dy = ini Sap fag) v|: (6.74) 


dim W=N 
W satisf. (6.12) 


As ae > dy, localized reduced basis methods generally result in larger Vy to sat- 
isfy a given approximation error tolerance £. Since we can represent any basis vector 
of a global reduced basis approximation of (6.11) with respect to the localizing space 
decomposition (6.10) as a sum of M'*' := M + #7,; + #7;¢ + #7; local vectors, we have 
the a priori bound doc, y < dy- In other words, if we denote by N (N glob) the number 
of reduced basis vectors required for a localized (global) reduced basis approximation 
for given £ and denoting by N loc the maximum dimension of the local reduced basis 


spaces Vý, V}, V, Vy, we have 
N < NM" < Ney" < Cz MN”, (6.75) 


where the constant Cz, only depends on the topology of the domain decomposition 
Ty. Whether or not estimate (6.75) is sharp largely depends on the dependence of the 
solution u(u) on the parameter u. When a change in p equally affects the solution in all 
subdomains Q, we expect that optimal local reduced basis spaces will be of similar 
dimension N' and that N'°° ~ N81, On the other hand, it may be the case that the 
influence of u on u(y) is weak in many Qm, in which case N « N loc yy'ot or that each 
of the p components of u € R” affects u(u) on different subdomains, in which case 
Niet < NEP M'®*, Thus, the actual loss in online efficiency due to localization will 
strongly depend on the type of problem to be solved. 

More importantly though, note that the localization of Vy results in a change of 
the structure of the reduced system matrices A1. While these matrices are dense for 
global reduced basis approximations, localized reduced basis schemes yield A? with 
a sparse block structure of M** x M' blocks of maximum dimension N! x N! and 
a maximum of Ceup blocks per row; Ceup depends on the specific localization method 
and on the topology of 7p. For instance, for nonconforming methods, Ceup — 1 is given 
by the maximum number of interfaces of a given subdomain Q, whereas for the Ar- 
biLoMod with a quadrilateral mesh Coup = 25. 

Thus, estimate (6.75) has to be interpreted in relation to the fact that the compu- 
tational complexity for solving (6.71) can be vastly reduced in comparison to (6.72) by 
exploiting the structure of A‘. In particular, the costs for assembling (6.71) can be re- 
duced to O(Cz, Coup(N loc)? 1). For the solution of (6.71) direct or block-preconditioned 
iterative solvers can be used. For the latter, the computational effort can be expected to 
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increase subquadratically in the number of subdomains M. In the scRBE method, the 
volume degrees of freedom associated with the spaces Vi are eliminated from (6.71) 
using static condensation to improve computational efficiency. 


6.7.2 Offline costs and parallelization 


While the local reduced basis spaces V,", Vi, Vý, Vy can be initialized by decom- 
posing global solution snapshots u,(u) with respect to (6.10) [5], the core element of 
localized reduced basis methods is the construction of local reduced basis spaces from 
local problems associated with the subdomains Qm, as described in Section 6.4. This 
has various computational benefits: 

First, we can expect a reduction of computational complexity as for most linear 
solvers we expect a superlinear increase in computational complexity for an increas- 
ing dimension of V;, whereas the ratio of the dimensions of V, and the local sub- 
spaces in (6.10) remains constant of order 1/M (for the volume spaces Vy and smaller 
for the spaces V”, V£, V¥) as h > 0. Thus, solving O(MN'®) training problems of size 
dim V,/M is expected to be faster than solving N global problems. At the same time, 
we expect N! to decrease for H — h. Thus smaller subdomains Qm will generally 
lead to shorter offline times at the expense of less optimal spaces Vy. In particular, for 
the nonconforming schemes in Section 6.3.2.2 it is readily seen that for H = h we have 
Vy = V, and that (6.13) and (6.11) are equivalent. 

Even more important than a potential reduction of complexity is the possibil- 
ity to choose H small enough such that each local training problem can be solved 
communication-free on a single compute node without the need for a high-perfor- 
mance interconnect. Also the problem setup and the computation of the reduced sys- 
tem (6.71) can be performed mostly communication-free: Instead of instantiating a 
global fine-scale compute mesh, each compute node can generate a local mesh from a 
geometry definition and solve training problems for a given local reduced basis space 
and all coupling spaces to obtain corresponding block-entries in A4, F1. Only the lo- 
cal geometry and the resulting reduced-order quantities are communicated (see [20, 
Section 8]). This makes localized reduced basis methods attractive for cloud-based en- 
vironments, where large computational resources can be dynamically made available, 
but communication speed is limited. 

Depending on the problem structure, the use of online enrichment (Section 6.6) 
can yield smaller, problem-adapted reduced spaces Vy. Similar to the training of Vy, 
online enrichment is based on the solution of small independent local problems that 
can easily be parallelized. As typically only some fraction of the subspaces of Vy un- 
dergo enrichment, fewer computational resources need to be allocated during an on- 
line enrichment phase. It has to be noted, however, that online enrichment leads to 
a propagation of snapshot data through the computational domain as the value of 
the current solution u,(y) at the boundary of the enrichment problem domain enters 
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the problem definition. Thus, to perform online enrichment, (boundary values of) re- 
duced basis vectors have to be communicated between compute nodes and the entire 
reduced basis has to be kept available. 


6.8 Applications and numerical experiments 


6.8.1 Multiscale problems 


We demonstrate the IP localized reduced basis methods from Section 6.3.2.2 in the con- 
text of parametric multiscale problems, such as Example 6.2, with a focus on online 
adaptivity as in Section 6.6 (using the a posteriori error estimate from Section 6.5.2), 
rather than offline training. These experiments were first published in the context of 
the online adaptive LRBMS in [90]. We consider a multiplicative splitting of the pa- 
rameter dependency and the multiscale nature of the data functions, in the sense that 
K(H) := AQwx,, with a parametric total mobility A : P — L” (Q) and a highly hetero- 
geneous permeability field x, € L™(Q)*4, To be more precise, we consider (6.2) on 
Q = [0,5] x [0, 1] with f(x, y) = 2-10? if (x,y) € [0.95, 1.10] x [0.30, 0.45], f(x, y) = -1-10? 
if (x,y) € [3.00, 3.15] x [0.75, 0.90] or (x,y) € [4.25, 4.40] x [0.25,0.40], and O every- 
where else, A(x, y; u) = 1+ (1-p)A,(x, y), homogeneous Dirichlet boundary values, and 
a parameter space P = [0.1,1]. On each t € Th, Kel¢ is the corresponding 0-th entry 
of the permeability tensor used in the first model of the 10th SPE Comparative Solu- 
tion Project (which is given by 100 x 20 constant tensors, see [104]) and A, models a 
channel, as depicted in Figure 6.6, top left. 

The right-hand side f models a strong source in the middle left of the domain and 
two sinks in the top and right middle of the domain, as is visible in the structure of the 
solutions (Figure 6.6, third row). The role of the parameter p is to toggle the existence 
of the channel A,. Thus A(1)x, = x, while u = 0.1 models the removal of a large con- 
ductivity region near the center of the domain (see the second row in Figure 6.6). This 
missing channel has a visible impact on the structure of the pressure distribution as 
well as the reconstructed velocities, as we observe in the last two rows of Figure 6.6. 
With a contrast of 10° in the diffusion tensor and an € of about |Q|/2,000, this setup is 
a challenging heterogeneous multiscale problem. 

We used several software packages for this numerical experiment and refer to [90] 
for a full list and instructions on how to reproduce these results. We would like to 
mention that all grid-related structures (such as data functions, operators, function- 
als, products, norms) were implemented in a DUNE-based C++ discretization (which 
is by now contained in the DUNE extension modules”? and the generic discretization 


10 https://github.com/dune-community/dune-xt/ 
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Figure 6.6: Data functions and sample solutions of the experiment in Section 6.8.1. First row: Loca- 
tion of the channel function A, (left) and plot of the force f (right) modeling one source (black: 2- 10°) 
and two sinks (dark gray: —1 - 107, zero elsewhere). Second to fourth rows: Both plots in each row 
share the same color map (middle) with different ranges per row, for parameters y = 1 (left column) 
and u = 0.1 (right column). From top to bottom: Logarithmic plot of A(u)k, (dark: 1.41 - 10-3, light: 
1.41 - 103), plot of the pressure u,(p) (IP localized FOM solution of (6.2), dark: -3.92 - 1074, light: 
7.61 - 10-1, isolines at 10%, 20%, 45%, 75%, and 95%), and plot of the magnitude of the recon- 
structed diffusive flux R? [u (0); u] (defined in (6.65) and (6.66), dark: 3.10 - 10°, light: 3.01 - 102). 
Note the presence of high-conductivity channels in the permeability (second row left, light regions) 
throughout large parts of the domain. The parameter dependency models a removal of one such 
channel in the middle right of the domain (second row right), well visible in the reconstructed Darcy 
velocity fields (bottom). 


toolbox dune-gdt!), while we used pyMOR [84] for everything related to model reduc- 
tion (such as Gram-Schmidt and greedy). We consider a domain decomposition of 
[Tal = 25 x 5 squares, each refined such that the full global grid would consist of 
[Tal = 1,014,000 elements. For the IP localized FOM, following Section 6.3.2.2, we 
choose on each subdomain Qn € Ty the discontinuous Galerkin space (first order), 
product, and bilinear form from Example 6.10. For error estimation, we employed the 
flux reconstruction ansatz from Section 6.5.2 using a zero-order diffusive flux recon- 
struction (compare Theorem 6.21). 

The sole purpose of these experiments is to demonstrate the capabilities of local- 
ized reduced basis methods regarding online enrichment. We thus initialize the local 
reduced spaces Vý on each subdomain a priori by orthonormalized Lagrangian shape 
functions of order up to one, thus obtaining a reduced space with poor approximation 
properties (comparable to a standard discontinuous Galerkin space with respect to 
the domain decomposition). Since we employ the a posteriori error estimate 7 on the 
full approximation error (including the discretization as well as the model reduction 


11 https://github.com/dune-community/dune-gdt/ 
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error) from Theorem 6.21, and since we omit grid refinement in these experiments, the 
estimated discretization error over all parameters of 1.66 is a lower bound for the over- 
all approximation error, and we thus choose a tolerance of Apnline = 2 for the online 
enrichment in Algorithm 6.3. 

We compare two different strategies, corresponding to the two plots in Figure 6.7. 
In both cases, we simulate an outer loop application in the online part by randomly 
choosing ten parameters Pontine © P which are subsequently processed. For each 
parameter, the local reduced spaces are enriched according to Algorithm 6.3 and the 
respective marking strategy, until the estimated error is below the specified toler- 
ance. Note that the evaluation of the localizable a posteriori error estimate can be 
fully offline-online decomposed and that after each enrichment only information 
from a subdomain and its neighbors are required to locally update the offline-online 
decomposed data. 
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Figure 6.7: Estimated error evolution during the adaptive online phase for the experiment in Sec- 
tion 6.8.1 with |7] = 125, ky = 1, Aoniine = 2 (dotted line), ñ = fa = 0.1, for different online 

and offline strategies: no global snapshot (greedy search disabled, Ngreegy = 0) during the offline 
phase, uniform marking during the online phase (top) and two global snapshots (greedy search on 
Pirain = {0.1,1}, Ngreedy = 2) and combined uniform marking while n(u, ñ, À) > OuniAontine with 
Ouni = 10, Dörfler marking with Odoerf = 0.85, and age-based marking with Nage = 4 (bottom left); 
note the different scales. With each strategy the local reduced bases are enriched according to Algo- 
rithm 6.3 while subsequently processing the online parameters 4o»... , Mo (bottom right). 
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In the first experiment, we use a uniform marking strategy, which results in an un- 
conditional enrichment on each subdomain (comparable to domain decomposition 
methods). As we observe in Figure 6.7 (top), however, it takes 129 enrichment steps 
to lower the estimated error below the desired tolerance for the first online param- 
eter Mọ. After this extensive enrichment it takes 12 steps for u, and none or one en- 
richment step to reach the desired tolerance for the other online parameters. The re- 
sulting coarse reduced space is of size 10,749 (with an average of 86 basis functions 
per subdomain), which is clearly not optimal. Although each subdomain was marked 
for enrichment, the sizes of the final local reduced bases differ since the local Gram- 
Schmidt basis extension may reject updates (if the added basis function is locally not 
linearly independent). As we observe in Figure 6.8 (left) this is indeed the case with 
local basis sizes ranging between 24 and 148. Obviously, a straightforward domain de- 
composition ansatz without suitable training is not feasible for this setup. This is not 
surprising since the data functions exhibit strong multiscale features and nonlocal 
high-conductivity channels connecting domain boundaries; see Figure 6.6. 


I) re . 


Figure 6.8: Spatial distribution of the final sizes of the local reduced bases on each subdomain, 
after the adaptive online phase for the experiment in Section 6.8.1 with Q = [0,5] x [0,1], |7y| = 
25 x 5 for the two strategies shown in Figure 6.7: no global snapshot with uniform enrichment (left, 
light: 24, dark: 148) and two global snapshots with adaptive enrichment (right, light: 9, dark: 20). 
Note the pronounced structure (right) reflecting the spatial structure of the data functions (compare 
Figure 6.6). 


To remedy the situation we allow for two global snapshots during the offline phase (for 
parameters u € {0.1, 1}) and use an adaptive marking strategy which combines uniform 
marking, Dörfler marking, and age-based marking (see the caption of Figure 6.7) in 
the online phase. This strategy employs uniform marking until a saturation condition 
is reached, and afterwards uses a Dörfler marking combined with a marking based 
on counting how often a subdomain has not been marked. With two global solution 
snapshots incorporated in the basis the situation improves significantly, as we observe 
in Figure 6.7 (bottom left). In total we observe only two enrichment steps with uniform 
marking (see the first two steps for uy), which indicates that further offline training 
would be beneficial. The number of elements marked ranges between 11 and 110 (over 
all online parameters and all but the first two enrichment steps) with a mean of 29 and 
a median of 22. Of these marked elements only once have 87 out of 110 elements been 
marked due to their age (see the last step for y,). Overall we could reach a significantly 
lower overall basis size than in the previous setup (1,375 vs. 10,749) and the sizes of 
the final local bases range between only 9 and 20 (compared to 24 to 148 above). We 
also observe in Figure 6.8 (right) that the spatial distribution of the basis sizes follows 
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the spatial structure of the data functions (compare Figure 6.6), which nicely shows 
the localization qualities of our error estimator. 


6.8.2 Fluid dynamics 


Flow simulations in pipelined channels have a growing interest in many biological 
and industrial applications. The localized model order reduction approaches pre- 
sented in this chapter are suitable for the study of internal flows in hierarchical 
parameterized geometries. In particular, the nonconforming approach introduced in 
Section 6.3.2 has applications in the analysis of blood flow in specific compartments 
of the circulatory system that can be represented as a combination of few deformed 
vessels from a reference one. 

We want to solve the Stokes equation defined in (6.3), with 6 = 0, in a computa- 
tional domain Q composed by two stenosed blocks Q, and Q,,, (Figure 6.9), by impos- 
ing nonhomogeneous boundary conditions o? = [0,5] in the inlet surface (x; = 10), 
nonhomogeneous boundary conditions on = [0,-1]7 in the outlet surface (x; = 0) 
and homogeneous Dirichlet boundary conditions on the remaining boundaries of the 
domain. Here, the Taylor-Hood FEM has been used to compute the basis functions, 
P, elements for velocity and supremizer (cf. Chapter 8 of Volume 3 of Model order 
reduction for a definition of supremizer functions), P, for pressure, respectively, and 
consequently P(T mm ) for the Lagrange multipliers space. Figure 6.11 shows the dis- 
tribution of the parameter values selected by the greedy algorithm, by applying the 
offline stage of the reduced basis method to the single stenosis block. By taking into 
account that the range [-5, 5] is not admitted, we can see that the higher concentration 
of values is in the intervals [—10, —5] and [5,10] in correspondence to larger deforma- 
tion of the pipe. 


Figure 6.9: Computational domain (44 = 7, fy = 10). 


The geometry of a single stenosis is obtained by the deformation of a reference pipe 
through a parameter that represents the contraction in the middle of the pipe. The 
deformed domain Q, is mapped from the straight reference pipe Ô of length L = 5 
and radius r = 1 through the following coordinate transformation T, : û —> Q, such 
asx = T(x) and x, = ĝ + 4 (cos( 7) -1),% =% + 2 (cos(42) - 1), X; = X3. The 
range of the parameter p is [—20, —5] U [5,20]; Figure 6.10 shows the reference pipe 
and some representative deformations of the geometry. In order to compute the basis 
functions, we consider a parameterized Stokes problem for each subdomain. For the 
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Figure 6.10: Reference pipe and two deformed pipes (u = —5, p = 5): stenosis and aneurysm configu- 
rations. 


5 4 Figure 6.11: Distribution of the selected param- 
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first subdomain, we compute the reduced basis imposing zero Dirichlet condition on 


the wall and Neumann boundary conditions given by imposing o,, = o -n = v% -— 
pn to be ao" = [0,5]" on Tin and o®"' = O on the internal interface. For the second 


subdomain, we compute the reduced basis imposing zero Dirichlet condition on the 
wall and Neumann boundary conditions imposing o” = 0 on the internal interface 
and 0°" = [0, -1]" on the outflow interface Tout- 

Moreover, we enrich the local reduced basis spaces by a coarse FE solution of the 
problem computed in the global domain. This strategy ensures not only the continuity 
of the velocity, but also the one of the normal stress along the internal interface. For 
this reason this method is called reduced basis hybrid method. Coarse and fine grids 
have been chosen in order to deal with respectively 155 and 2,714 nodes in a single 


block domain. Figure 6.12 shows a representative flow solution in Q, found with the 


Figure 6.12: Representative solutions of velocity using the reduced basis hybrid method (with N, = 
N, = 19) (left) and using the FEM as a global solution (right), 44 = 7, p> = 10. 


6 Localized model reduction for parameterized problems —— 295 


Figure 6.13: Representative solutions of pressure using the reduced basis hybrid method (with N, = 
Nz = 19) (left) and using the FEM as a global solution (right), p = 7, p2 = 10. 


reduced basis hybrid method, to be compared with the FE solution. The same compar- 
ison, regarding the pressure solutions, is shown in Figure 6.13. 


6.9 Further perspectives 


6.9.1 Parabolic problems 


Most of the techniques presented in this chapter so far can be extended or even di- 
rectly applied to parabolic problems. For instance, local approximation spaces that 
are optimal in the sense of Kolmogorov are proposed in [99] and the LRBMS method 
for parabolic problems is presented in [87, 86]. To facilitate an adaptive construction 
of the local reduced space or online adaptivity, a suitable, localized a posteriori error 
estimator is key. Therefore, we present in this subsection an abstract framework for a 
posteriori error estimation for approximations of scalar parabolic evolution equations, 
based on elliptic reconstruction techniques. For further reading and the application 
to localized model reduction we refer to [44, 87]. 


Definition 6.22 (Parameterized parabolic problem in variational form). Let a Gelfand 
triple of suitable Hilbert spaces V c H = H’ c V’, an end time T,,,4 > O, initial data 
Uy € V, and a right-hand side f € H be given. For a parameter u € P find u(; u) € 
L°(0, Teng; V) with 0,u(-;H) € L7(0, Teng; V’), such that u(0; u) = uo and 


(dult; p), q) + a(u(tsH), v3) = f(v;p) forallve V. (6.76) 


Depending on the error we want to quantify, the space V in (6.76) can be either an 
analytical function space as in (6.1) or an already discretized function space V;,. We 
drop the parameter dependency in this section to simplify the notation. 


Definition 6.23 (Approximations of the parabolic problem). Let V c H be a finite- 
dimensional approximation space for V, not necessarily contained in V. Potential 
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candidates for V are conforming or nonconforming localized model reduction spaces 
Vy as discussed above, but also FE or FV spaces fit into this setting. Denote by (-,-), 
|| - || the H-inner product and the norm induced by it. 

Let f ¢ H, and let a, : (V + V) x (V + V) > R be a discrete bilinear form which 
coincides with a on V x V and is thus continuous and coercive on V. Let further ||l-|| 
be a norm over V + V, which coincides with the square root of the symmetric part of 
ap over V. 

Our goal is to bound the error e(t) := u(t)-ŭ(t) between the analytical (or discrete) 
solution u € L°(0, Tena; V), ju € L7(0, Teng; V’) of (6.76), where the duality pairing 
(d,u(t), v) is induced by the H-scalar product via the Gelfand triple and the V-Galerkin 
approximation it € L?(0, Teng» V), dù € L7(O, Teng V), solution of 


(0,i1(t), 7) + a, (a(t), 7) = (f,0) forallv V. (6.77) 


Definition 6.24 (Elliptic reconstruction). Denote by II the H-orthogonal projection 
onto V. For Ÿ € V, define the elliptic reconstruction R,(7) € V of Ÿ to be the unique 
solution of the variational problem 


an(Rey(V),v) = (A, (V) - TF) +f,v) forallv eV, (6.78) 


where A; (V) € T is the H-inner product Riesz representative of the functional ay(¥,-), 
i.e., (A,(V), 1’) = ap, v’) for all 7’ € V. Note that R,(V) is well-defined, due to the 
coercivity of a, on V. 


From the definition it is clear that 7 is the V-Galerkin approximation of the elliptic 
reconstruction Ra). 

Let us assume that for each t we have a decomposition ii(t) =: i°(t)+ ütt) (not nec- 
essarily unique), where ùf (t) € V, u(t) € V are the conforming and nonconforming 
parts of a(t). We consider the following error quantities: 


p(t) := u(t) - Rey (a0), ElI) = Rey (ùlt)) - àlt), 
e(t) := u(t) - a(t), E (É) = Rey lit) - WO. 


Theorem 6.25 (Abstract semi-discrete error estimate). Let C := (2y? + 1)"°, where yp 
denotes the continuity constant of a, on V with respect to ||l-|||. Then 


lellzz¢o.r,,aitay She COD + Vt liota 


sa 
+ (C + 1) - lello, Taam + E ME leor amm 


Note that ¢(t) denotes the approximation error of the coercive variational prob- 
lem (6.78). Hence, this error contribution can be controlled by invoking any (local- 
ized) a posteriori error estimate for coercive variational problem as, e. g., presented in 
Section 6.5. 

It is straightforward to modify the estimate in Theorem 6.25 for semi-discrete so- 
lutions u(t) to take the time discretization error into account. 


6 Localized model reduction for parameterized problems —— 297 


Corollary 6.26. Let ù € 17(0,TengV), Oi € LO, Teng V) be an arbitrary discrete 
approximation of u(t), not necessarily satisfying (6.77). Let R[p](t) € V denote the 
V-Riesz representative with respect to the H-inner product of the time-stepping residual 
of u(t), i.e., 


(Rrit) V) = (00,0) + a7) - F4) We. 
Then, with C := (3y? + 2)"°, the following error estimate holds: 


lelro, m < NEON + Ai lzor a 


~d 
+(C+1)-llellzo,r,,gsthy + © M lezor anm 
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6.9.2 Nonaffine parameter dependence and nonlinear problems 


A key ingredient towards model order reduction for nonlinear problems is the empiri- 
calinterpolation method introduced in [13] and further developed in [33], [26, 75]. For a 
general exposition we refer to Chapter 5 of this volume of Model order reduction. 

In the context of localized model order reduction, empirical interpolation has 
been employed in, e. g., [22, 96, 86]. Based on the concept of empirical operator in- 
terpolation from [33], localization strategies can be employed as follows. To present 
the main ideas, let us assume the simple situation that 


M 
V, =V 
m=1 
and that we have a localized decomposition as follows: 


Ap (Up (p) Vas) = Z% ax (u (u), vi; u), 


with a;,(u,(H),3H) € (V;)’. The strategy will then rely on an empirical operator in- 
terpolation of the local volume operators aj"(uj'(H),;#) € (V;")' and will thus only 
involve localized computations in the construction of the interpolation operator. As 
an example, the interpolation of the local volume operator will be of the form 


L 
TT [ay (up). 3B] = > "(an (up QD) 3B) a7" 


=1 


for a local collateral basis ta Ha c ry and corresponding interpolation function- 
als {.Y/"}i., c Xp" from a suitable local dictionary 27" c (V;")", the choice of which 
is crucial to ensure the accuracy as well as an online-efficient evaluation of the inter- 
polant. Note that because the isomorphism between V;" and is bi-dual, the local dic- 
tionary of interpolation functionals a can be identified with a dictionary of func- 


tions ry" c Vj", such that <7" (a uM u), sh) = ax (uiu), oj"; u), where oj” e 2M 
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corresponds to .“; ic € Ea An online-efficient evaluation of the interpolated oper- 
ator Z7" [a} (uj, (H), ; H)] can be ensured by choosing the local dictionary £}' such that 
the computational complexity of the evaluation a} (u}' (y), 0"; p) for o™ € Xẹ' does not 
depend on the dimension of V;”. The choice of 2; thus depends on the underlying 
discretization: Possible choices in the context of FE schemes include the FE basis of 
Vi’. Other choices of 2" are conceivable and could improve the interpolation quality, 
which is subject to further investigation. 


6.10 Conclusion 


In this chapter we have given an introduction to projection-based localized model 
order reduction techniques for parameterized coercive problems. Starting from an 
abstract localization framework, we have presented conforming and nonconforming 
coupling schemes for the local reduced spaces within this unified framework. For the 
generation of the local reduced spaces we have discussed a priori approaches based 
on polynomial spaces at the domain interfaces, as well as empirical approaches based 
on randomized localized training as an approximation of the optimal local reduced 
spaces. Further, we have introduced rigorous localizable a posteriori error estimates 
and discussed their use to steer an adaptive online enrichment of the approximation 
spaces based on the solution of local correction problems. Finally, we have discussed 
the online efficiency and parallelizability of the presented methods. We have given 
application examples from the fields of multiscale problems, linear elasticity, and 
fluid dynamics. Extensions to time-dependent or nonlinear problems are under active 
research, and we have given a brief outlook towards localized model order reduction 
for these problem classes in the final section of this chapter. 
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Steven L. Brunton and J. Nathan Kutz 
7 Data-driven methods for reduced-order 
modeling 


Abstract: Data-driven mathematical methods are increasingly important for charac- 
terizing complex systems across the physical, engineering, and biological sciences. 
These methods aim to discover and exploit a relatively small subset of the full high- 
dimensional state space where low-dimensional models can be used to describe the 
evolution of the system. Emerging dimensionality reduction methods, such as the dy- 
namic mode decomposition (DMD) and its Koopman generalization, have garnered at- 
tention due to the fact that they can (i) discover low-rank spatio-temporal patterns of 
activity, (ii) embed the dynamics in the subspace in an equation-free manner (i.e., 
the governing equations are unknown), unlike Galerkin projection onto proper or- 
thogonal decomposition modes, and (iii) provide approximations in terms of linear 
dynamical systems, which are amenable to simple analysis techniques. The selec- 
tion of observables (features) for the DMD/Koopman architecture can yield accurate 
low-dimensional embeddings for nonlinear partial differential equations (PDEs) while 
limiting computational costs. Indeed, a good choice of observables, including time 
delay embeddings, can often linearize the nonlinear manifold by making the spatio- 
temporal dynamics weakly nonlinear. In addition to DMD/Koopman decompositions, 
coarse-grained models for spatio-temporal systems can also be discovered using the 
sparse identification of nonlinear dynamics (SINDy) algorithm which allows one to 
construct reduced-order models in low-dimensional embeddings. These methods can 
be used in a nonintrusive, equation-free manner for improved computational perfor- 
mance on parametric PDE systems. 


Keywords: reduced-order model, dynamic mode decomposition, Koopman theory, 
nonlinear system identification, sparse regression, system identification 


PACS: 02.30.Hq, 02.30.Jr, 02.60.Cb, 02.30.Mv 


7.1 Introduction 


Data-driven modeling of complex systems is of increasing importance in modern 
scientific applications given the unprecedented rise of data collection on multi- 
scale, spatio-temporal systems. Enabled by emerging sensor technologies and high- 
performance computing platforms, the large-scale monitoring and collection of data 
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on such systems has shifted our modeling paradigm by exploiting data-driven, ma- 
chine learning approaches. Specifically, instead of positing empirical or approxi- 
mate spatio-temporal models, typically characterized by partial differential equations 
(PDEs), the low-dimensional features extracted from time snapshots of the data can be 
directly used to construct reduced-order models (ROMs) for a variety of important tasks 
such as state-space reconstruction and diagnostics, as well as future state prediction 
and forecasting [14]. In this chapter, we present a diverse set of data-driven methods 
that can be used to construct ROMs directly from data. The methods presented can 
be used with traditional ROM architectures where the governing PDEs are known, or 
they can be used to discover unknown spatio-temporal dynamics directly from the 
data. Most of the methods are nonintrusive, minimizing the need for prohibitively 
expensive high-performance simulations. This also allows for accurate, low-fidelity 
models, enabling inexpensive Monte Carlo simulations. We present four methods for 
enabling data-driven ROMs: The dynamic mode decomposition (DMD) and the asso- 
ciated Koopman decomposition [78], the sparse identification of nonlinear dynamics 
(SINDy) algorithm [25], and the Hankel alternative view of Koopman (HAVOK) algo- 
rithm [22]. Each method can be used to advantage in a variety of situations, including 
when the governing PDE equations are known, only partially known, or unknown. 

Consider a governing system of nonlinear PDEs of a single spatial variable x, 
which can be modeled as [14] 


u; = L(x)u + N(u, x,t), (71) 


where L(x) is a linear operator and N(-) prescribes the nonlinear terms in the evolution 
dynamics. Both L(x) and N(-) may be unknown, or only partially known. As an exam- 
ple, the Burgers equation, u, = Uyy + vuu,, has L = 0°/dx” and N(u) = vuu,. Associated 
with (7.1) are a set of initial and boundary conditions on a domain x € D. Historically, 
a number of analytic solution techniques have been devised to study (7.1) provided the 
right-hand side is known. Typically the aim of such methods is to reduce the PDE (7.1) 
to a set of ordinary differential equations (ODEs). The standard PDE methods of sep- 
aration of variables and similarity solutions are constructed for this express purpose. 
Once in the form of an ODE, a broader variety of analytic methods can be applied along 
with a qualitative theory in the case of nonlinear behavior [61]. This again highlights 
the role that asymptotics can play in characterizing behavior. 

For the general form of (7.1) where the right-hand side is known, separation of 
variables can often be used to yield a computational algorithm capable of producing 
low-rank approximations. Since the spatial solutions are not known a priori, it is typi- 
cal to assume a Set of basis modes which can be used for the low-rank approximation. 
Indeed, such assumptions on basis modes underly the critical ideas of the method of 
eigenfunction expansions. This yields a separation of variables solution ansatz of the 
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form 


u(x, t) = Palt) = X yrat), (7.2) 


k=1 


where W(x) € C”" form aset of r orthonormal basis modes and x € R”! represents the 
spatial discretization of x in the governing PDE. The modal basis is often obtained 
via proper orthogonal decomposition (POD) [60, 14, 129]. This separation of variables 
solution approximates the true solution, provided r is large enough. A fundamental 
assumption of reduced-order modeling is that there exists a low-rank truncation, or 
subspace, that accurately characterizes the evolution of the spatio-temporal system. 
More broadly, such approximations are based upon modal methods for building ROMs 
as discussed in Chapters 1 and 4 of Volume 1 of Model order reduction [12]. 

The orthogonality properties of the basis functions w,(x), which are the columns 
of Y, enable us to make use of (7.2). Inserting the expansion (7.2) into the governing 
equations gives [14] 

da T T 
ae Y’ LYa + Y N(Pa). (7.3) 
The given form of N(-) determines the mode-coupling that occurs between the various r 
modes. Indeed, the hallmark feature of nonlinearity is the production of modal mixing 
from (7.3). Equation (7.3) is the canonical ROM identified as a Galerkin projection of the 
dynamics onto POD modes. It can be evaluated given full knowledge of the right-hand 
side of the governing PDE. 

Equation (7.3) details how a low-rank subspace can be used to construct a 
Galerkin-POD-ROM model as a proxy, or surrogate, model for the high-fidelity model. 
In this reduction, the linear operator 7 LY can be computed once to produce an r xr 
matrix modeling the effects of the linear portion of the dynamics. What is more prob- 
lematic is the evaluation of the nonlinear contribution w'N(Pa) in (7.3). Indeed, one of 
the primary challenges in producing the low-rank dynamical system is efficiently pro- 
jecting the nonlinearity of the governing PDEs on the POD basis. This fact was recog- 
nized early on in the ROM community, and methods such as gappy POD [50, 142, 150] 
were proposed to more efficiently enable this hyperreduction task. More recently, the 
empirical interpolation method (EIM) [11], the discrete EIM (DEIM) [37], and the QR 
decomposition-based Q-DEIM [46], have provided a computationally efficient method 
for discretely (sparsely) sampling and evaluating the nonlinearity. These widely used 
hyperreduction methods ensure that the computational complexity of ROMs scale 
favorably with the rank of the approximation, even for complex nonlinearities. 

Numerical schemes based on the Galerkin projection (7.3) are commonly used to 
perform simulations of the full governing system (7.1). Convergence to the true solution 
can be accomplished by judicious choice of both the modal basis elements ¥ and the 
total number of modes r. Interestingly, the separation of variables strategy, which is 
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rooted in linear PDEs, works for nonlinear and nonconstant coefficient PDEs, provided 
enough modal basis functions are chosen in order to capture the nonlinear mode mix- 
ing that occurs in (73). A good choice of modal basis elements allows for a smaller set 
of r modes to be chosen to achieve a desired accuracy. The POD method is designed to 
specifically address the data-driven selection of a set of basis modes that are tailored 
to the particular dynamics, geometry, and parameters. 

Unfortunately, the Galerkin-POD projection of the dynamics (7.3) is often unsta- 
ble [34], requiring modification to stabilize the time-stepping scheme [4]. Moreover, 
the evaluation in (7.3) of the nonlinear term ¥7N(a) renders the ROM scheme intru- 
sive, i.e., to compute the nonlinear contribution in the low-rank subspace requires an 
expensive sampling of the high-fidelity model in order to build the low-rank subspace. 
Instead, one can directly approximate the nonlinearity via DMD which directly com- 
putes this contribution via nonintrusive methods [2]. Thus there is no recourse to high- 
fidelity and expensive computations to construct an approximation to the nonlinear 
contribution. If latent variables are present, i. e., important portions of the state-space 
have not been measured, then the Hankel alternative view of Koopman (HAVOK) al- 
gorithm, which helps to discover a proxy for the latent variable space, can be used 
instead of DMD. Finally, if the right-hand side is unknown, then the SINDy algorithm 
can be used to discover a low-rank, nonlinear model characterizing the evolution. The 
diversity of mathematical techniques highlights the emerging use of regression and 
machine learning strategies that can help model complex, spatio-temporal systems. 


7.2 Data-driven reductions 


Numerical linear algebra plays a central role in scientific computing [135, 77, 24]. 
Specifically, methods that have historically improved the efficiency of solving Ax = b 
have always been of critical importance for tractable computations, especially at 
scale, where scale is a relative term associated with the limits of computing in a given 
era. From QR decomposition to lower-upper factorization [135], matrix decomposi- 
tions have been the primary methods to enable improved computational efficiency. 
But perhaps the most important factorization technique is the singular value decom- 
position (SVD) [77], which plays a key role in data analysis and computation, including 
applications in reduced-order modeling through POD and DMD. 


7.2.1 Singular value decomposition 


The success of the SVD algorithm is largely due to the fact that by construction, it 
extracts the dominant, correlated features from any given data matrix. This often al- 
lows one to approximate the matrix with a principled low-rank approximation which 
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is guaranteed to be the best approximation in an @,-sense. This mathematical archi- 
tecture is so powerful and universal that it has been invented and used extensively in 
a wide range of fields [77]. Specifically, it is alternatively known as principal compo- 
nent analysis (PCA) in statistics (where to be precise, each column or row is scaled to 
have mean zero and unit variance), POD in the fluid dynamics community, empirical 
mode decomposition in atmospheric sciences, the Hotelling transform, empirical eigen- 
functions, or Karhunen-Loéve decomposition. Thus, from seemingly complex data from 
which a matrix is composed, a low-dimensional subspace can be computed on which 
the majority of the data resides. 
The SVD of a matrix X € C”™” takes the form 


X = UZV“, (7.4) 


in terms of the following three matrices: 


U e Cc" is unitary, (75a) 
V e C™™ is unitary, (7.5b) 
X € R™" is diagonal. (7.5c) 


Additionally, it is assumed that the diagonal entries of X are nonnegative and ordered 
from largest to smallest so that 0; > 0) >---> Op 2 0, where p = min(m, n). The SVD of 
the matrix X thus shows that the matrix first applies a unitary transformation preserv- 
ing the unit sphere via V*. This is followed by a stretching operation that creates an 
ellipse with principal semi-axes given by the matrix X. Finally, the generated hyperel- 
lipse is rotated by the unitary transformation U. Thus the image of a unit sphere under 
any n x m matrix is a hyperellipse. The following is the primary theorem concerning 
SVD [135]. 


Theorem. Every matrix X € C™™ has an SVD (7.4). Furthermore, the singular values 
{0;} are uniquely determined, and if X is square and g; is distinct, the singular vectors 
{u,} and {v;} are uniquely determined up to complex signs (complex scalar factors of 
absolute value 1). 


The above theorem guarantees the existence of the SVD, but in practice, it still 
remains to be computed. This is a fairly straightforward process if one considers the 
following matrix products: 


X*X = (UEV*)"(UZV*) = Vz°Vv" (76) 
and 


XX* = (UZV*)(UZV*)" = UZ°U". (77) 
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Multiplying (7.6) and (7.7) on the right by V and U, respectively, gives the two self- 
consistent eigenvalue problems 


X*XV = VY’, (7.8a) 
XX*U = Ux’. (7.8b) 


Thus if the normalized eigenvectors are found for these two equations, then the or- 
thonormal basis vectors are produced for U and V. Likewise, the square root of the 
eigenvalues of these equations produces the singular values oj. 


Theorem (Schmidt-Eckart-Young—Mirsky theorem [118, 47, 96]). For any N so that 
O < N <p = min{m, n}, we can define the partial sum 


N 
Xy = > ouv? f (7.9) 
jal 


And if N = min{m, n}, we define oy, = 0. Then 
IX — Xyllz = Ons1- (7.10) 


Likewise, if using the Frobenius norm, then 


IX—Xyllp = Jor, + OR 9 +--+. (711) 

The interpretation of this theorem is critical as it gives a geometrical perspective 
for understanding the SVD. Geometrically, the SVD gives the best approximation of a 
hyperellipsoid by a line segment, i.e., simply take the line segment to be the longest 
axis, i.e., that associated with the singular value o,. Continuing this idea, what is the 
best approximation by a two-dimensional ellipse? Take the longest and second longest 
axes, i. e., those associated with the singular values g; and o>. After r steps, the total 
energy in X is completely captured. Thus the SVD gives an algorithm fora least-squares 
fit allowing us to project the matrix onto low-dimensional representations in a formal, 
algorithmic way. 

The SVD provides a principled way to find a low-rank subspace on which to project 
the evolution dynamics of the PDE in (7.1). Specifically, the first r modes of a low-rank 
projection form the POD basis in (7.2) desired for model reduction 


WY =U,. (7.12) 


These basis modes are used to project the dynamics onto the dominant, low-rank sub- 
space of activity as shown in (7.3). Of course, to use these POD modes, the dynamics 
of the governing PDE must be known in advance. Moreover, the Galerkin projection of 
the dynamics onto POD modes in (7.3) may be, depending on the underlying problem, 
unstable, requiring modification to stabilize the time-stepping scheme. Such stability 
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issues have been considered extensively by Carlberg and co-workers [34], Amsallem 
and Farhat [4], and Kalashnikova et al. [67]. Regardless, POD reductions arising from 
the SVD computation of the basis ¥ form the underpinnings of many ROMs [60, 14]. 
A significant advantage of the maturity of POD-based reductions is the ability to pro- 
duce rigorous error bounds. Indeed, there is a rich literature on how to use the error 
properties of POD/SVD to derive rigorous error bounds for simulation as well as opti- 
mal control across a diverse set of applications [75, 76, 143, 110, 59, 136]. Such rigorous 
bounds provide trust regions and certifiable models for many critical application ar- 
eas. Details of POD-based ROMs can be found in Chapter 2 of the current volume on 
Model order reduction [13]. 

Due to tremendous advances and innovations, modern large-scale simulations 
and/or the data collection process can quickly produce volumes of data that tradi- 
tional methods could not easily analyze and diagnose in real-time. This emerging 
big data era requires diagnostic tools that can scale to meet the rapidly growing in- 
formation acquired from new monitoring technologies which are producing increas- 
ingly fine-scale spatial and temporal measurements. In such cases, one can exploit 
new techniques that are capable of extracting the dominant global features of the 
high-dimensional dynamics. Specifically, emerging randomized linear algebra algo- 
rithms [55, 85, 48] are critically enabling for scalable big data applications, supple- 
menting the method of snapshots [122] for efficient computation of the SVD. Random- 
ized algorithms exploit the fact that the data themselves have low-rank features. In- 
deed, the method scales with the intrinsic rank of the dynamics rather than the dimen- 
sion of the measurements/sensor space. This is in contrast to standard SVD/PCA/POD 
reductions which do not scale well with the data size. One can think of the scalable 
methods as being critically enabling for producing real-time analysis of emerging, 
streaming big data sets. Moreover, the dominant features of the data can be used for 
an efficient compression of the data for storage or reduced-order modeling applica- 
tions [3]. Figure 7.1 outlines the basic algorithmic architecture which can be used for 
producing scalable SVD decompositions. 


7.2.2 Dynamic mode decomposition 


An alternative to POD is the DMD reduction [117]. Unlike POD, the DMD algorithm not 
only correlates spatial activity, but also enforces that various low-rank spatial modes 
be correlated in time, essentially merging the favorable aspects of POD/PCA in space 
and the Fourier transform in time. DMD is a matrix factorization method based upon 
the SVD algorithm. However, in addition to performing a low-rank SVD approxima- 
tion, it further performs an eigendecomposition on a best-fit linear operator that ad- 
vances measurements forward in time in the computed subspaces in order to extract 
critical temporal features. Thus the DMD method provides a spatio-temporal decom- 
position of data into a set of dynamic modes that are derived from snapshots or mea- 
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Figure 7.1: Illustration of the randomized matrix decomposition technique for scalable decomposi- 
tions. The random sampling matrix Q is used to produce a new matrix Y which can be decomposed 
using a QR decomposition. This leads to the construction of the matrix B which is used for approxi- 
mating the left and right singular vector. From Erichson etal. [48]. 


surements of a given system in time, arranged as column state-vectors. The mathemat- 
ics underlying the extraction of dynamic information from time-resolved snapshots 
is closely related to the idea of the Arnoldi algorithm, one of the workhorses of fast 
computational solvers. The DMD algorithm was originally designed to collect data at 
regularly spaced intervals of time. However, new innovations allow for both sparse 
spatial [27, 54] and temporal [139] collection of data as well as irregularly spaced col- 
lection times [6]. 

Like SVD, the DMD algorithm is based upon a regression. Thus there are a vari- 
ety of algorithms that have been proposed in the literature for computing the DMD. A 
highly intuitive understanding of the DMD architecture was proposed by Tu et al. [138], 
which provides the exact DMD method. 


Definition: Exact dynamic mode decomposition (Tu et al. 2014 [138]). Suppose we 
have a dynamical system (7.1) and two sets of measurement data 


X= u Uw > Unale (7.13a) 


X'=|u uy - uha (713b) 
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so that u, = F(u,), where F is the map corresponding to the evolution of (71) for 
time At. Exact DMD computes the leading eigendecomposition of the best-fit linear 
operator A relating the data u’ ~ Au: 


A=X'X'. (7.14) 


The DMD modes, also called dynamic modes, are the eigenvectors of A, and each DMD 
mode corresponds to a particular eigenvalue of A. 


The DMD framework takes an equation-free perspective where the original, non- 
linear dynamics may be unknown. Thus measurements of the system alone are used 
to approximate the dynamics and predict the future state. However, DMD can also be 
used when governing equations are known [2]. This DMD-Galerkin procedure repre- 
sents a potential hybrid between the POD-Galerkin and DMD methods. The integration 
of DMD and POD can also be used for model reduction numerical schemes [146]. The 
DMD procedure constructs a proxy, locally linear dynamical system approximation 
to (7.1): 


Uk+1 = Aux, (7.15) 
whose well-known solution is 
n 
ur = X PAKb; = @A'd, (746) 
jel 


where œ; and A; are the eigenvectors and eigenvalues of the matrix A, and the co- 
efficients b; are the coordinates of the initial condition uy in the eigenvector basis. 
The eigenvalues A of A determine the temporal dynamics of the system, at least in 
an asymptotic sense and for normal operators, i.e., transient dynamics are not well 
captured. It is often convenient to convert these eigenvalues to continuous time, w = 
log(A)/At, so the real parts of the eigenvalues w determine growth and decay of the so- 
lution, and the imaginary parts determine oscillatory behaviors and their correspond- 
ing frequencies. The eigenvalues and eigenvectors are critically enabling for produc- 
ing interpretable diagnostic features of the dynamics. It is important to note that the 
choice of the time step At is critical in the DMD algorithm. The time step must be small 
enough to resolve the fastest time scales of relevance. A consequence of the linear 
model produced by the DMD algorithm is its inability to model transient phenomena 
over the snapshots sampled, aside from transient growth potentially produced by non- 
normal modes where eigenvalues are identical or nearly so. 

The DMD algorithm produces a low-rank eigendecomposition of the matrix A 
that optimally fits the measured trajectory u% for k = 1,2,...,m snapshots in a least- 
squares sense so that ||u;,,, — Auzll2 is minimized across all points for k = 1,2,...,m-1. 
The optimality of the approximation holds only over the sampling window where A 
is constructed, and the approximate solution can be used to not only make future 
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state predictions, but also to derive dynamic modes critical for diagnostics. Indeed, 
in much of the literature where DMD is applied, it is primarily used as a diagnostic 
tool. This is much like POD analysis, where the POD modes are also primarily used for 
diagnostic purposes. Thus the DMD algorithm can be thought of as a modification of 
the SVD architecture which attempts to account for dynamic activity of the data. The 
eigendecomposition of the low rank space found from SVD enforces a Fourier mode 
time expansion which allows one to then make spatio-temporal correlations with the 
sampled data. Recently, DMD has also been rigorously connected to the spectral POD 
method [133]. 

Early variants of the DMD-computed eigenvalues that were biased by the presence 
of sensor noise [58, 44]. This was a direct result of the fact that the standard algorithms 
treated the data in a pairwise sense and favored the forward direction in time. Dawson 
etal. [44] and Hemati et al. [58] developed several methods for debiasing within the 
standard DMD framework. These methods have the advantage that they can be com- 
puted with essentially the same set of robust and fast tools as the standard DMD. As 
an alternative, the optimized DMD advocated by [38] treats all of the snapshots of the 
data at once. This avoids much of the bias of the original DMD but requires the solution 
of a potentially large nonlinear optimization problem. Askham and Kutz [6] recently 
showed that the optimized DMD algorithm could be rendered numerically tractable by 
leveraging the classical variable projection method [53]. Moreover, the optimized DMD 
method can be used to enforce all eigenvalues to have a real part less than or equal to 
zero. This ensures stability of solutions for future times as there are no growing modes. 
For input-output systems, DMD has also been modified through a postprocessing al- 
gorithm to generate a stable input-output model [15]. These methods show that DMD 
architectures can be imbued with advantageous stability properties for ROMs. 

The variable projection algorithm is based upon the observation that the desired 
solutions of DMD are exponentials (7.16). Thus DMD is reformulated as an exponential 
data fitting (specifically, for inverse differential equations), an area of research that has 
been extensively developed and has many applications [52, 104]. The variable projec- 
tion method leverages the special structure of the exponential data fitting problem, so 
that many of the unknowns may be eliminated from the optimization. An additional 
benefit of these tools is that the snapshots of data no longer need to be taken at regu- 
lar intervals, i. e., the sample times do not need to be equispaced. The goal is then to 
rewrite the data matrix of snapshots as 


X' = D(a)B, (7.17) 
where @(a) € C™" with entries defined by Pla); j = exp(a;t)). 
The preceding leads us to the following definition of the optimized DMD in terms 


of an exponential fitting problem. Suppose that & and Ê solve 


minimize|X' - ®(@)B\|, overa € C*,B € Cc. (718) 
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The optimized DMD eigenvalues are then defined by A; = â; and the eigenmodes are 
defined by 


1 


ia tle A, (719) 
IB‘ DIL 


Pi 


where Ê'(;, i) is the i-th column of B’. Details of the algorithm and code for computing 
the optimized DMD can be found in Askham and Kutz [6]. The improved and debiased 
decomposition (7.16) of this optimal DMD strategy are readily apparent in numerous 
examples. Moreover, a comparison of DMD variants shows how each method handles 
noise and takes on bias. Optimized DMD thus far outperforms all other variants at the 
cost of a nonlinear optimization. 

A remarkable feature of the DMD algorithm is its modularity for mathematical en- 
hancements. Specifically, the DMD algorithm can be engineered to exploit sparse sam- 
pling [27, 54], it can be modified to handle inputs and actuation [106], it can be used 
to more accurately approximate the Koopman operator when using judiciously cho- 
sen functions of the state-space [80], and it can easily decompose data into multiscale 
temporal features in order to produce a multiresolution DMD [79]. Few mathematical 
architectures are capable of seamlessly integrating such diverse modifications of the 
dynamical system. But since the DMD provides an approximation of a linear system, 
such modifications are easily constructed. Moreover, the DMD algorithm, unlike many 
other machine learning algorithms, is not data-intensive in comparison to most deep 
neural network architectures which require large labeled data sets. Thus a DMD ap- 
proximation can always be achieved, especially as the first step in the algorithm is the 
SVD which is guaranteed to exist for any data matrix. However, for very large data sets, 
DMD can leverage randomized methods [55, 85, 48] to produce a scalable randomized 
DMD [49, 18]. 

DMD is closely related to the field of system identification, which identifies mod- 
els from data, often for use with model-based controllers. Tu etal. [138] and Proctor 
etal. [106] established connections between DMD and several classical system identi- 
fication approaches, including the eigensystem realization algorithm [64] and singu- 
lar spectrum analysis (SSA) [20] in climate time-series analysis. Nearly all methods of 
system identification involve some form of regression of data onto dynamics, and the 
main distinction between the various techniques is the degree to which this regression 
is constrained. For example, DMD generates best-fit linear models. 


7.2.3 Koopman theory and observable selection 


Much of the challenge associated with predicting, estimating, controlling, and reduc- 
ing complex systems arises from the inherent nonlinearity in the governing equations. 
Indeed, mathematical physics has a rich history in deriving coordinate transforma- 
tions that simplify the dynamics and alleviate the challenge of nonlinearity. In 1931, 
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Koopman developed an alternative perspective to classical dynamical systems theory, 
showing that there is a linear, infinite-dimensional operator that acts on the Hilbert 
space of possible measurement functions of the system, advancing these measure- 
ments along the flow of the dynamics [71, 72]. Koopman’s operator-theoretic perspec- 
tive trades nonlinear dynamics for linear but infinite-dimensional dynamics, and was 
critical in Birkhoff’s proof of the ergodic theorem [17, 97]. 

Recently, Koopman operator theory has seen a resurgence of interest [93, 29, 94], 
in large part because of the increasing availability of measurement data and improving 
computational capabilities. In 2005, Mezic showed that Koopman theory may be used 
to provide a modal decomposition of complex systems, providing direct relevance to 
engineering systems [93]. Since then, it has been shown that the DMD algorithm from 
fluid dynamics [117] actually approximates the Koopman operator [109], restricted toa 
set of linear measurements of the system; a more detailed treatment for fluid systems 
is given by Taira et al. [129]. 

The ability of Koopman analysis to transform nonlinear systems into a linear 
framework has tremendous promise to make complex systems amenable to optimal 
prediction, estimation, and control with simple techniques from linear systems theory. 
In a short time, Koopman theory has been extended to nonlinear estimation [125, 126] 
and control [106, 107], for example via model predictive control [73, 66], control in 
eigenfunction coordinates [65], and switching control [103]. However, Koopman the- 
ory appears to follow the principle of conservation of difficulty, in that finding the 
right nonlinear measurements that enable a tractable linear representation may be as 
challenging as solving the original problem. In a sense, obtaining Koopman embed- 
dings may be seen as an expensive offline computation that enables fast and efficient 
online prediction, estimation, and control. In addition, the Koopman operator is 
one of two main candidates for analyzing a dynamical system using operator-based 
approaches, the other being the Perron—Frobenius operator. The Perron—Frobenius 
operator evolves probability density functions along the flow of the dynamics, while 
the Koopman operator evolves observable functions of the state. These two operators 
are adjoint to each other in appropriately defined function spaces and it should there- 
fore theoretically not matter which one is used to study the system’s behavior [70]. 

Before introducing the mathematical formulation of Koopman operator theory, we 
first consider the flow map Fx obtained by integrating the PDE in (7.1) for a short-time 
At, given by 


Uk = FaU). (7.20) 
The Koopman operator K is defined so that 


Kig =8°F, (7.21) 
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where is the composition operator. For a discrete-time system with time step At, this 
becomes 


Kug (Ux) = 8(Fae(uj)) = (Uy). (7.22) 


In other words, the Koopman operator defines an infinite-dimensional linear dynam- 
ical system that advances the observation of the state g; = g(u,) to the next time step: 


guk) = Kar (Uy). (7.23) 


Note that this is true for any observable function g and for any state ux. 

Much of the challenge of modern Koopman theory is obtaining a finite-dimen- 
sional representation K of the infinite-dimensional operator K. In practice, this 
amounts to discovering eigenfunctions of the Koopman operator, which are measure- 
ment functions that behave linearly when evolved forward in time. A discrete-time 
Koopman eigenfunction (u) corresponding to eigenvalue A satisfies 


P(Uks1) = Kupur) = Apu). (7.24) 


In continuous-time, a Koopman eigenfunction (u) satisfies 


< pu) = Kolu) = Ag(u). (7.25) 


Obtaining Koopman eigenfunctions from data or from analytic expressions is a central 
applied challenge in modern dynamical systems. Discovering these eigenfunctions 
enables globally linear representations of strongly nonlinear systems. Applying the 
chain rule to the time derivative of the Koopman eigenfunction (u) yields 


d 
ae = Vp(u) -ù = Ve(u) - f(u). (7.26) 
Combined with (7.25), this results in a PDE for the eigenfunction (u): 
Ve(u) - f(u) = Ag(u). (7.27) 


With this nonlinear PDE, it is possible to approximate the eigenfunctions, either by 
solving for the Laurent series or with data via regression, both of which are explored 
below. This formulation assumes that the dynamics are both continuous and differen- 
tiable. The discrete-time dynamics in (7.20) are more general, although in many exam- 
ples the continuous-time dynamics have a simpler representation than the discrete- 
time map for long times. Koopman analysis has recently been extended to the con- 
tinuous PDE formulation, rather than just the high-dimensional discretized ODE con- 
text, for example, showing that the Cole—Hopf transform is a Koopman embedding for 
Burgers’ equation [80]. 
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There are many approaches to obtain finite-dimensional approximations to the 
Koopman operator. DMD is a representation based on linear observables [109], which 
has been extended to nonlinear observables in the extended DMD (eDMD) [144] and 
the variational approach of conformation dynamics [99, 100]. In all of these cases, 
it is important that the measurements are chosen to form a Koopman-invariant sub- 
space [23]; otherwise, the projection of the Koopman operator onto this subspace will 
result in spurious eigenvalues and eigenfunctions. 

In eDMD, an augmented state is constructed: 


0 (u) 


65(u) 
y=ou=| >. |. (7.28) 


0u) 


The projection © may contain the original state u as well as nonlinear measurements, 
so often p > n. Next, two data matrices are constructed, as in DMD: 


| 


Y= ]¥1 Yo e Ym|> Yo=Hl¥o V3 e Ymal- (7.29a) 


Finally, a best-fit linear operator Ay is constructed that maps Y into Y’: 
Ay = argming, |Y - AyY|, = Y'Y’. (7.30) 
This regression may be written in terms of the data matrices @(X) and O(X’): 
. Try! T Try! T t 
Ay = argminą, |0" (X') - Ay®' (X)||, = @ (X')(®'(X)) . (7.31) 


The resulting nonlinear model for ux is given by the proxy eDMD variable y;,.,, = AyYx.- 
Because the augmented vector y may be significantly larger than the state u, kernel 
methods are often employed to compute this regression [145]. In principle, the en- 
riched library © provides a larger basis in which to approximate the Koopman op- 
erator. It has been shown recently that in the limit of infinite snapshots, the eDMD 
operator converges to the Koopman operator projected onto the subspace spanned by 
O [144, 70, 74]. However, if © does not span a Koopman-invariant subspace, then the 
projected operator may not have any resemblance to the original Koopman operator, 
as all of the eigenvalues and eigenvectors may be different. In fact, it was shown that 
the eDMD operator will have spurious eigenvalues and eigenvectors unless it is repre- 
sented in terms of a Koopman-invariant subspace [23]. Therefore, it is essential to use 
validation and cross-validation techniques to ensure that DMD models are not over- 
fit, as discussed below. For example, it was shown that eDMD cannot contain the orig- 
inal state u as a measurement and represent a system that has multiple fixed points, 
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periodic orbits, or other attractors, because these systems cannot be topologically con- 
jugate to a finite-dimensional linear system [23]. Recently, researchers have been lever- 
aging the representational power of deep neural networks to identify Koopman eigen- 
functions and approximate Koopman operators [130, 149, 92, 141, 101, 83]. In the next 
section, we will discuss an alternative approach to obtain a Koopman-invariant sub- 
space based on time delay coordinates [22]. 


7.2.4 Time-delay embeddings for Koopman embeddings 


Instead of advancing instantaneous linear or nonlinear measurements of the state of 
a system directly, as in DMD, it may be possible to obtain intrinsic measurement co- 
ordinates for Koopman based on time-delayed measurements of the system [127, 22, 
5, 43, 68]. This perspective is data-driven, relying on the wealth of information from 
previous measurements to inform the future. Unlike a linear or weakly nonlinear sys- 
tem, where trajectories may get trapped at fixed points or on periodic orbits, chaotic 
dynamics are particularly well suited to this analysis: Trajectories evolve to densely fill 
an attractor, so more data provide more information. The use of delay coordinates may 
be especially important for systems with long-term memory effects, where the Koop- 
man approach has recently been shown to provide a successful analysis tool [128]. 
Interestingly, a connection between the Koopman operator and the Takens embed- 
ding was explored as early as in 2004 [95], where a stochastic Koopman operator is 
defined and a statistical Takens theorem is proven. One version of time-delay embed- 
dings, the HAVOK, has been used successfully to diagnose a diverse set of dynamical 
systems [22]. More broadly, there are a number of analysis tools that can be applied to 
the Hankel matrix for analysis of dynamics [68]. 

The time-delay measurement scheme is shown schematically in Figure 7.2, as illus- 
trated on the Lorenz system for a single time-series measurement of the first variable, 
x(t). If the conditions of the Takens embedding theorem are satisfied [131], it is possi- 
ble to obtain a diffeomorphism between a delay-embedded attractor and the attractor 
in the original coordinates. We then obtain eigentime-delay coordinates from a time 
series of a single measurement x(t) by taking the SVD of the Hankel matrix H: 


X(t) X(t) Xm) 
x(t) x(t3) Pos X(t +1) 

= ` į x = WrpzV", (7.32) 
X(tm,) X(tm,+1) 94 X(tm) 


where m, is the number of snapshots and m, is the total number of delays. The 
columns of yp and V from the SVD are arranged hierarchically by their ability to 
model the columns and rows of H, respectively. Often, H may admit a low-rank ap- 
proximation by the first r columns of Yp and V. Note that the Hankel matrix in (7.32) is 
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Figure 7.2: Schematic of the Hankel alternative view of Koopman (HAVOK) algorithm [22], as illus- 
trated on the Lorenz 63 system. A time series x(t) is stacked into a Hankel matrix H. The SVD of H 
yields a hierarchy of eigentime series that produce a delay-embedded attractor. A best-fit linear re- 
gression model is obtained on the delay coordinates v; the linear fit for the first r — 1 variables is 
excellent, but the last coordinate v, is not well modeled as linear. Instead, v, is an input that forces 
the first r — 1 variables. Rare forcing events correspond to lobe switching in the chaotic dynamics. 
From Brunton and Kutz [24], modified from [22]. 


the basis of the eigensystem realization algorithm [64] in linear system identification 
and SSA [20] in climate time-series analysis. 

The low-rank approximation to (7.32) provides a data-driven measurement system 
that is approximately invariant to the Koopman operator for states on the attractor. 
By definition, the dynamics map the attractor into itself, making it invariant to the 
flow. In other words, the columns of U form a Koopman-invariant subspace. We may 
rewrite (7.32) with the Koopman operator K £ Kyt: 


x(t) Kx(t,)) «KM y(t) 
Kx(t)) Kx) +++ K™ex(t) 

H= ' , , , . (7.33) 
K™ x(t) Kex) e K™ x(t) 


The columns of (7.32) are well approximated by the first r columns of W yp. The first r 
columns of V provide a time series of the magnitude of each of the columns of Yp 
in the data. By plotting the first three columns of V, we obtain an embedded attractor 
for the Lorenz system (Figure 7.2). 

The connection between eigentime-delay coordinates from (7.32) and the Koop- 
man operator motivates a linear regression model on the variables in V. Even with 
an approximately Koopman-invariant measurement system, there remain challenges 
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to identifying a linear model for a chaotic system. A linear model, however detailed, 
cannot capture multiple fixed points or the unpredictable behavior characteristic of 
chaos with a positive Lyapunov exponent [23]. Instead of constructing a closed linear 
model for the first r variables in V, we build a linear model on the first r — 1 variables 
and recast the last variable, v,, as a forcing term: 


u(t = Av(t) + Bv,(t), (734) 


wherev=[v; v> = val] T isavectorofthefirstr -1 eigentime-delay coordinates. 
Other work has investigated the splitting of dynamics into deterministic linear and 
chaotic stochastic dynamics [93]. 

In all of the examples explored in [22], the linear model on the first r — 1 terms is 
accurate, while no linear model represents v,. Instead, v, is an input forcing to the lin- 
ear dynamics in (7.34), which approximates the nonlinear dynamics. The statistics of 
v,(t) are non-Gaussian, with long tails corresponding to rare-event forcing that drives 
lobe switching in the Lorenz system; this is related to rare-event forcing distributions 
observed and modeled by others [86, 113, 87]. 


7.3 Data-driven model discovery 


For many modern complex systems of interest, such as in materials science, neuro- 
science, epidemiology, climate science, and finance, there is a basic lack of physi- 
cal laws and governing equations. Even when governing equations are available, for 
example in fluid turbulence, protein folding, and combustion, the equations are so 
complex that they are not readily amenable to analysis. With increasingly complex 
systems, and the emergence of powerful computing architectures and big data, the 
paradigm is shifting to data-driven methods to discover governing equations [19, 119, 
25, 111]. 


7.3.1 SINDy: sparse identification of nonlinear dynamics 


Discovering ROMs from data is a central challenge in modern computational physics. 
Typically, the form of a candidate model is either constrained via prior knowledge of 
the governing equations, as in Galerkin projection [98, 9, 34], or a handful of heuris- 
tic models are tested and parameters are optimized to fit data. Alternatively, best-fit 
linear models may be obtained using DMD. Simultaneously identifying the nonlinear 
structure and parameters of a model from data is considerably more challenging, as 
there are combinatorially many possible model structures. 
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The SINDy algorithm [25] bypasses the intractable combinatorial search through 

all possible model structures, leveraging the fact that many dynamical systems 
d 

—a = f(a 7.35 
gef (7.35) 
have dynamics f with only a few active terms in the space of possible right-hand 
side functions; for example, the Lorenz equations (Figure 7.3) only have a few linear 
and quadratic interaction terms per equation. Here, a € R” is a low-dimensional 
state, for example obtained via POD/SVD [25, 81], or constructed from physically re- 
alizable measurements, such as lift, drag, and the derivative of lift for aerodynamic 
systems [82]. 
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Figure 7.3: Schematic of the sparse identification of nonlinear dynamics (SINDy) algorithm [25], as 
illustrated on the Lorenz 63 system. From Brunton and Kutz [24], modified from [25]. 


We then seek to approximate f by a generalized linear model in a set of candidate basis 
functions 6;(a) 


Pp 
f(a) = È 0, (a& = O(a)E, (7.36) 
k=1 


with the fewest nonzero terms in E. It is possible to solve for the relevant terms that 
are active in the dynamics using sparse regression [132, 155, 57, 63] that penalizes the 
number of terms in the dynamics and scales well to large problems. 

First, time-series data are collected from (7.35) and formed into a data matrix: 


T 
A=[a(t,) alt) ---a(tn)] - (737) 
A similar matrix of derivatives is formed: 
; : ? . T 
A=[a(t) a(t) ---a(t,)] - (7.38) 
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In practice, this may be computed directly from the data in A using a numerical differ- 
encing scheme, for instance. However, for noisy data, the total-variation regularized 
derivative tends to provide numerically robust derivatives [36]. Alternatively, it is pos- 
sible to formulate the SINDy algorithm for discrete-time systems a;,,; = F(a), as in 
the DMD algorithm, and avoid derivatives entirely. 

A library of candidate nonlinear functions O(A) may be constructed from the data 
in A: 


@(A)=[1 A A’ o A? o sin(A) ol. (7.39) 


Here, the matrix Af denotes a matrix with column vectors given by all possible time 
series of d-th-degree polynomials in the state a. In general, this library of candidate 
functions is only limited by one’s imagination, but polynomials, trigonometric func- 
tions, and other well-known functions are a good starting point. The matrix 0 tends to 
be ill-conditioned as the library elements, such as polynomials, are often not orthogo- 
nal. Indeed, they can often be nearly aligned over the time course where the library is 
evaluated. Despite the high condition number, the sparse selection advocated below 
is able to identify the correct dynamics provided the noise level is sufficiently small. 

The dynamical system in (7.35) may now be represented in terms of the data ma- 
trices in (7.38) and (7.39) as 


A= O(A)E. (7.40) 


Each column ğ, in & is a vector of coefficients determining the active terms in the 
k-th row in (7.35). A parsimonious model will provide an accurate model fit in (7.40) 
with as few terms as possible in &. Such a model may be identified using a convex 
€,-regularized sparse regression: 


&, = argminy lÅ% - O(A)E ll + AF kh. (7.41) 


Here, A, is the k-th column of A and A is a sparsity-promoting regularization weight, 
typically chosen by simple hyperparameter tuning. Sparse regression, such as the 
LASSO [132] or the sequential thresholded least-squares (STLS) algorithm used in 
SINDy [25], improves the numerical robustness of this identification for noisy overde- 
termined problems, in contrast to earlier methods [140] that used compressed sens- 
ing [45, 30, 32, 31, 33, 10, 137]. We advocate STLS to select active terms; there are recent 
guarantees on the convergence of this algorithm [152], and it has also been formalized 
in a general sparse regression framework called SR3 [154]. 
The sparse vectors ¢, may be synthesized into a dynamical system: 


a, = O(a)é;,. (7.42) 


Note that x; is the k-th element of a and @(a) is a row vector of symbolic functions of 
a, as opposed to the data matrix O(A). 
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The result of the SINDy regression is a parsimonious model that includes only 
the most important terms required to explain the observed behavior. The sparse re- 
gression procedure used to identify the most parsimonious nonlinear model is a con- 
vex procedure. The alternative approach, which involves regression onto every possi- 
ble sparse nonlinear structure, constitutes an intractable brute-force search through 
the combinatorially many-candidate model forms. SINDy bypasses this combinato- 
rial search with modern convex optimization and machine learning. It is interesting 
to note that for discrete-time dynamics, if O(A) consists only of linear terms, and if we 
remove the sparsity promoting term by setting A = 0, then this algorithm reduces to 
DMD [117, 109, 138, 78]. If a least-squares regression is used, as in DMD, then evena 
small amount of measurement error or numerical round-off will lead to every term in 
the library being active in the dynamics, which is nonphysical. A major benefit of the 
SINDy architecture is the ability to identify parsimonious models that contain only the 
required nonlinear terms, resulting in interpretable models that avoid overfitting. 


7.3.1.1 Extensions and applications 


Because SINDy is formulated in terms of linear regression in a nonlinear library, it 
is highly extensible. The SINDy framework has been recently generalized by Loiseau 
and Brunton [81] to incorporate known physical constraints and symmetries in the 
equations by implementing a constrained sequentially thresholded least-squares op- 
timization. In particular, energy-preserving constraints on the quadratic nonlineari- 
ties in the Navier-Stokes equations were imposed to identify fluid systems [81], where 
it is known that these constraints promote stability [86, 9, 34]. This work also showed 
that polynomial libraries are particularly useful for building models of fluid flows in 
terms of POD coefficients, yielding interpretable models that are related to classical 
Galerkin projection [25, 81]. Loiseau et al. [82] also demonstrated the ability of SINDy to 
identify dynamical systems models of high-dimensional systems, such as fluid flows, 
from a few physical sensor measurements, such as lift and drag measurements on the 
cylinder. SINDy has also been applied to identify models in nonlinear optics [123] and 
plasma physics [40]. For actuated systems, SINDy has been generalized to include in- 
puts and control [26], and these models are highly effective for model predictive con- 
trol [66]. It is also possible to extend the SINDy algorithm to identify dynamics with 
rational function nonlinearities [88], with integral terms [116], and based on highly 
corrupt and incomplete data [134]. SINDy was also recently extended to incorporate 
information criteria for objective model selection [89], and to identify models with hid- 
den variables using delay coordinates [22]. Finally, the SINDy framework was general- 
ized to include partial derivatives, enabling the identification of PDE models [111, 115], 
which will be explored below. 

More generally, the use of sparsity-promoting methods in dynamics is quite 
recent [140, 114, 102, 84, 28, 105, 8, 7, 21, 90, 91]. Other techniques for dynami- 
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cal system discovery include methods to discover equations from time series [39], 
equation-free modeling [69], empirical dynamic modeling [124, 148], modeling emer- 
gent behavior [108], the nonlinear autoregressive model with exogenous inputs (NAR- 
MAX) [51, 153, 16, 121], and automated inference of dynamics [120, 41, 42]. Broadly 
speaking, these techniques may be classified as system identification, where methods 
from statistics and machine learning are used to identify dynamical systems from 
data. Nearly all methods of system identification involve some form of regression of 
data onto dynamics, and the main distinction between the various techniques is the 
degree to which this regression is constrained. For example, DMD generates best-fit 
linear models. Recent nonlinear regression techniques have produced nonlinear dy- 
namic models that preserve physical constraints, such as conservation of energy. Yao 
and Bollt previously formulated the dynamical system identification problem as a 
similar linear inverse problem [147], although sparsity-promoting regression was not 
used, so the resulting models included all terms in ©. In addition, SINDy is closely 
related to NARMAX [16], which identifies the structure of models from time-series data 
through an orthogonal least-squares procedure. 


7.3.2 Model discovery for PDEs 


A major extension of the SINDy modeling framework generalized the library to include 
partial derivatives, enabling the identification of PDEs [111, 115]. The resulting algo- 
rithm, called the PDE functional identification of nonlinear dynamics (PDE-FIND), 
shown in Figure 74, has been demonstrated to successfully identify several canoni- 
cal PDEs from classical physics, purely from noisy data. These PDEs include Navier— 
Stokes, Kuramoto-Sivashinsky, Schrödinger, reaction diffusion, Burgers, Korteweg- 
de Vries (KdV), and the diffusion equation for Brownian motion [111]. 

PDE-FIND is similar to SINDy, in that it is based on sparse regression in a library 
constructed from measurement data. PDE-FIND is outlined below for PDEs in a single 
variable, although the theory is readily generalized to higher dimensional PDEs. The 
spatial time-series data are arranged into a single column vector Y € C™”, represent- 
ing data collected over m time points and n spatial locations. Additional inputs, such 
as aknown potential for the Schrédinger equation, or the magnitude of complex data, 
is arranged into a column vector Q € C", Next, a library @(Y,Q) € C™™®? of D can- 
didate linear and nonlinear terms and partial derivatives for the PDE is constructed. 
Derivatives are taken either using finite differences for clean data, or when noise is 
added, with polynomial interpolation. The candidate linear and nonlinear terms and 
partial derivatives are then combined into a matrix O(Y, Q) which takes the form 


oxo- Y P... Q... YX YY, aL (7.43) 


Each column of © contains all of the values of a particular candidate function across 
all of the mn space-time grid points on which data are collected. The time derivative 
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Figure 7.4: Schematic of PDE-FIND [111], as illustrated on the fluid flow past a circular cylinder. From 
Rudy etal. [111]. 


Y; is also computed and reshaped into a column vector. As an example, a column of 
@(Y, Q) may be qu?. 
The PDE evolution can be expressed in this library as follows: 


Y, = O(Y, Q)E. (7.44) 


Each entry in € is a coefficient corresponding to a term in the PDE, and for canonical 
PDEs, the vector is sparse, meaning that only a few terms are active. 

If the library © has a sufficiently rich column space that the dynamics are in its 
span, then the PDE should be well represented by (7.44) with a sparse vector of coef- 
ficients €. To identify the few active terms in the dynamics, a sparsity-promoting re- 
gression is employed, as in SINDy. Importantly, the regression problem in (744) may 
be poorly conditioned. Errors in computing the derivatives will be magnified by nu- 
merical errors when inverting ©. Thus a least-squares regression radically changes 
the qualitative nature of the inferred dynamics. 

In general, we seek the sparsest vector ë that satisfies (744) with a small resid- 
ual. Instead of an intractable combinatorial search through all possible sparse vec- 
tor structures, a common technique is to relax the problem to a convex ¢,-regularized 
least-squares [132]; however, this tends to perform poorly with highly correlated data. 
Instead, we use ridge regression with hard thresholding, which we call sequential 
threshold ridge regression. For a given tolerance and threshold A, this gives a sparse 
approximation to €. 

We iteratively refine the tolerance of Algorithm 1 to find the best predictor based 
on the selection criteria, 


& = argming|O(Y, QE - Y; + ex(@(Y, Q))lEllo, (7.45) 
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where x(O) is the condition number of the matrix O, providing stronger regularization 
for ill-posed problems. Penalizing ||é||) discourages overfitting by selecting from the 
optimal position in a Pareto front. While in general this problem is NP-hard we are 
restricting it to solutions generated via the STRidge algorithm, which promotes hard 
thresholding. Such hard thresholding has been recently shown to be a proxy for the 
€o-norm [154]. 

As in the SINDy algorithm, it is important to provide sufficiently rich training data 
to disambiguate between several different models. For example, if only a single trav- 
eling wave from the KdV equation is analyzed, the method incorrectly identifies the 
standard linear advection equation, as this is the simplest equation that describes a 
single traveling wave. However, if two traveling waves of different amplitudes are ana- 
lyzed, the KdV equation is correctly identified, as it describes the different amplitude- 
dependent wave speeds [111]. 

The PDE-FIND algorithm can also be used to identify PDEs based on Lagrangian 
measurements that follow the path of individual particles. For example, it is possible 
to identify the diffusion equation describing Brownian motion of a particle based ona 
single long time-series measurement of the particle position. In this example, the time 
series is broken up into several short sequences, and the evolution of the distribution 
of these positions is used to identify the diffusion equation [111]. 


7.4 Data-driven ROMs 


The methods detailed in the previous sections can be integrated with traditional 
model reduction architectures. In what follows, we highlight how such methods can 
be used in a data-driven way to construct ROM models in a nonintrusive, efficient 
manner. 


7.4.1 Application of DMD and Koopman to ROM models 


DMD provides an alternative approach to computing the projection of the nonlinearity 
onto the rank-r POD subspace in (7.3). Specifically, instead of using POD modes and 
gappy sampling for approximation of the nonlinear, low-rank contribution to the dy- 
namics, DMD is used to directly compute a time evolution of the nonlinearity W'N(Pa) 
from snapshot data. Like the DEIM interpolation procedure [37], the DMD algorithm 
will proceed by constructing a snapshot matrix of the nonlinearity: 


Xn = N, N, KIRN Nin > (7.46) 
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where the columns N} = N(u(t,),X,t,) € C” are evaluations of the nonlinearity at 
time t ke 

Following (7.16), a DMD of the matrix Xy;, gives a low-rank approximation of the 
form 


N(u(f), X, t) = Dy, exp(Qy, by à (7.47) 


This low-rank approximation is achieved directly with further recourse to gappy in- 
terpolation for projecting back the DMD modes. The approximation can be used to 
modify (7.3) so as to achieve the following low-rank model: 
da T T 

u Y LYa + Y Dyr exp(Qy, t)byz. (7.48) 
This integration of POD and DMD methods has been shown to provide performance 
increases in comparison to POD alone [2]. Moreover, the technique can be integrated 
with randomized linear algebra decomposition methods to achieve further enhance- 
ments in computational speed and scalability. Alla and Kutz further show that the 
POD-DMD integration competes well with POD with DEIM in terms of accuracy, while 
significantly outperforming it in terms of computation time. The DMD algorithm itself 
is faster than POD with DEIM and POD-DMD, but suffers from poor accuracy. One can 
also envision using a DMD-DMD reduction whereby a projection-based reduction with 
DMD-Galerkin is performed along with a hyperreduction with DMD. DMD-based ROM 
models have also recently been successfully demonstrated in a number of technical 
applications [1, 56]. A more detailed analysis of interpolation methods can be found 
in Chapter 7 of Volume 1 of Model order reduction [12]. 


7.4.2 Application of SINDy for ROMs 


The SINDy algorithm can also be used to construct ROM architectures (7.2) from data 
alone, i. e., no governing equations are known a priori. As an example, the flow past 
a cylinder (Figure 7.5) provides a model with a rich history in fluid mechanics and 
dynamical systems [98]. It has long been theorized that turbulence is the result of a 
series of Hopf bifurcations that occur as the flow velocity increases [112], giving rise 
to more rich and intricate structures in the fluid. After 15 years, the first Hopf bifur- 
cation was discovered in a fluid system, in the transition from a steady laminar wake 
to laminar periodic vortex shedding at Reynolds number 47 [62, 151]. This discovery 
led to a long-standing debate about how a Hopf bifurcation, with cubic nonlinearity, 
can be exhibited in a Navier-Stokes fluid with quadratic nonlinearities. After 15 more 
years, this was resolved using a separation of time scales and a mean-field model by 
Noack et al. [98]. It was shown that coupling between oscillatory modes and the base 
flow gives rise to a slow manifold, resulting in algebraic terms that approximate cubic 
nonlinearities on slow time scales. 
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Figure 7.5: The vortex shedding past a cylinder is a prototypical example in fluid dynamics, with rele- 
vance to many applications, including drag reduction behind vehicles. Vortex shedding is the result 
of a Hopf bifurcation. However, because the Navier-Stokes equations have quadratic nonlinearity, it 
is necessary to employ a mean-field model with a separation of time scales, where a fast mean-field 
deformation is slave to the slow vortex shedding dynamics. The parabolic slow manifold is shown 
(left), with the unstable fixed point (C), mean flow (B), and vortex shedding (A). A POD basis and 
shift mode are used to reduce the dimension of the problem (middle right). The identified dynamics 
closely match the true trajectory in POD coordinates, and they capture the quadratic nonlinearity 
and time scales associated with the mean-field model. From Brunton, Proctor and Kutz [25]. 


This example provides a compelling test case for the proposed ROM-SINDy algorithm, 
since the underlying form of the dynamics took nearly three decades for experts in 
the community to uncover. Because the state dimension is large, it is advantageous 
to reduce the dimension of the system. POD provides a low-rank basis resulting in a 
hierarchy of orthonormal modes that, when truncated, capture the most energy of the 
original system for the given rank truncation. The first two most energetic POD modes 
capture a significant portion of the energy, and steady-state vortex shedding is a limit 
cycle in these coordinates. An additional mode, called the shift mode, is included to 
capture the transient dynamics connecting the unstable steady state with the mean of 
the limit cycle [98]. 

In the dominant POD coordinate system (rank r = 3), the mean-field model a = 
f(a) for the cylinder dynamics is discovered by SINDy to be [25]: 


å, = Ua, — wa, + Aa,a3, (749a) 
Ay = WA, + Hy + Aaza3, (749b) 
a; = -A(a; - a? - a2). (749c) 


Note that the governing equations for a(t) in (7.49) are closely related to the slow- 
manifold formulation of Noack et al. [98] formulated using the standard Galerkin-POD 
projection. Specifically, it discovers the correct model a = f(a) with quadratic non- 
linearities and reproduces a parabolic slow manifold. The a3 variable corresponds to 
the shift-mode of Noack et al. [98], and if A is large, so that the a;-dynamics are fast, 
then the mean flow rapidly corrects to be on the slow manifold a, = aj + a3 given 
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by the amplitude of vortex shedding. When substituting this algebraic relationship 
into equations (7.49a) and (7.49b), we recover the Hopf normal form on the slow mani- 
fold. Note that derivative measurements are not available, but are computed from the 
state variables. When the training data do not include trajectories that originate from 
the slow manifold, the algorithm incorrectly identifies cubic nonlinearities, and fails 
to identify the slow manifold. This model was subsequently improved by Loiseau and 
Brunton [81] to incorporate energy-conserving constraints and to include higher-order 
terms to model the effect of truncated POD modes. 


7.4.3 Application of time-delay embeddings for ROMs 


Time-delay embedding for building ROMs can be used in a completely data-driven ar- 
chitecture where the governing equations are unknown, or for building a Koopman 
operator for a known governing evolution equation [22]. Indeed, one can use time- 
delay embedding with the SINDy architecture when short time-delay embeddings are 
used, or for producing a direct Koopman approximation when long time-delay embed- 
dings are used. Champion etal. [35] highlight the various architectures possible. The 
short-time and long-time embedding possibilities are detailed here. 


7.4.3.1 Short time-delay embedding 


For a short time-delay embedding, the time-shifted data can provide a more accurate 
assessment of the true rank of the underlying system. Such time-delay embedding was 
used by Tu et al. [138] in order to ensure that the data were not rank-deficient. Indeed, 
without time-shifting the data, the DMD approximation does not capture the correct 
complex eigenvalue pairs associated with the periodic (Fourier) time dynamics. 

Figure 7.6 shows the effects of the time-delay embedding as illustrated on the sim- 
ple Van der Pol oscillator. In the top left panel of this figure, the singular values of H 
given by (7.32) for a short time-delay embedding is shown. Specifically, the data were 
delayed by five time steps. For this delay, the rank of the matrix H is dominated by 
two modes. The time dynamics of the first three modes are shown in the middle panel 
of the figure, illustrating the strongly nonlinear Van der Pol oscillations. A reduced 
model can then be constructed from the first two modes so that Wyp spans a rank 
2 subspace. Importantly, the dominant nonlinear time-series data can then be used 
with the SINDy architecture to discover the governing equations and build a dynami- 
cal ROM model. 
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Figure 7.6: Time-delay embedding of the Van der Pol oscillator with time steps of 0.01. (a) With a 
short time-delay embedding of five time steps, the SVD produces a dominant low-rank (two-mode) 
truncation whose time-dynamic modes are illustrated in (c). (b) With a long time-delay embedding 
of several hundred time steps, the SVD produces a low-rank truncation of approximately a dozen 
modes whose time dynamic modes are illustrated in (d). Note that the short time-delay modes are 
strongly nonlinear oscillatory modes while the long time delay produces nearly perfect sinusoidal 
modes. Details can be found in Champion, Brunton, and Kutz [35]. 


7.4.3.2 Long time-delay embedding 


For long time-delay embeddings, the nonlinear dynamics can be made to be approxi- 
mately linear, thus providing an approximation to the Koopman operator and a linear 
ROM. The long time-delay embedding is especially useful in a data-driven architec- 
ture where the governing equations are unknown. Moreover, the time-delay embed- 
ding can significantly improve upon the DMD algorithm for producing an approximate 
dynamical system for forecasting. 

Figure 7.6 shows the effects of the time-delay embedding as illustrated on the sim- 
ple Van der Pol oscillator. In the top right panel of this figure, the singular values 
of H for a long-time delay embedding are shown. Specifically, the data were delayed 
by several hundred time steps which spanned more than a period of the nonlinear 
oscillations. Unlike the short time-delay embedding, the rank increases from two to 
about a dozen. The time dynamics of the first three of these dozen modes (i. e., the first 
three columns of the V matrix of (7.32)) are shown in the bottom panel. Note that the 
time modes with the long delay are now approximately sinusoidal, thus being ideal 
for a DMD/Koopman approximation. In this case, the SINDy architecture is unneces- 
sary. 
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7.5 Conclusion and outlook 


ROMs continue to play a critically enabling role in emulation and simulation strate- 
gies. Indeed, ROMs are making many intractable computations tractable by providing 
a surrogate model that can be computed at a fraction of the cost and with improved 
memory constraints. For emerging models in multiscale dynamical systems, such asin 
biology, atmospheric dynamics, and molecular dynamics simulations, ROMs provide 
a scalable mathematical framework, where it is possible to obtain accurate statistical 
estimates of the properties of the high-fidelity model from low-fidelity models. 

Data-driven approaches to ROMs are also playing an increasingly important role 
in developing scalable and nonintrusive emulators. Thus the governing equations, 
which may be unknown or only partially known, can be approximated by a suite 
of emerging mathematical methods. Table 7.1 highlights the various methods that 
are available for producing data-driven ROMs. They are compared to the standard 
Galerkin-POD architecture. Importantly, for each ROM architecture, two things must 
be prescribed in the underlying separation of variable strategy (7.2): (i) the subspace 
on which the ROM is to be constructed, and (ii) the manner of extracting the dynamical 
evolution in this subspace. Of course, such reductions do not guarantee the construc- 
tion of a stable ROM model, as recently highlighted by Carlberg etal. [34]. Thus for 
each ROM model strategy, care must be taken in order to produce a stable, low-rank 
emulator. Indeed, both POD-Galerkin and POD-DMD algorithms, for instance, must 
be modified in order to promote a stable time-stepping ROM. 

If the governing evolution equations (7.1) are known, then a Galerkin-POD (or 
Petrov—Galerkin-POD) provides a simple projective method for producing a ROM. One 
can also use the DMD algorithm in this architecture (POD-DMD) for more rapid eval- 
uation of the nonlinear terms. For unknown governing equations where the full state 


Table 7.1: Model reduction algorithms and their subspaces. Included is one example reference high- 
lighting the method. 


Data-driven ROM algorithms 
ROM model u(x, t) = ‘¥(x)a(t) 


Galerkin-POD [14] à = Y'LYa + Y'N(Ya) 

DMD [78] u = Mexp(Ot)b 

POD-DMD [2] à = Y'LYa + Y’ Oy exp(Qy ft) by, 

POD-SINDy [25] a = f(a) dynamics on subspace ¥ 
HAVOK-SINDy [35] a = f(a) dynamics on subspace yp (short delay) 
HAVOK-Koopman [22] a = Ka dynamics on subspace ¥-p (long delay) 
Basis elements (rank r) 

POD modes X = [u] uy ... u,,] = VZV" 

DMD modes X = [u, Uy ... Um] = Pexp(Ot)b 

nonlinear DMD modes N= [N; N; ... Nm] = Py, exp(Qyt)by, 
Time-delay Koopman modes H = ‘P7pXV 
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space is sampled, DMD can be used to produce a low-rank, best-fit linear model for 
the dynamics. An alternative to DMD is the POD-SINDy algorithm, which discovers 
a low-rank, nonlinear dynamical system approximating the dynamics of the system. 
Time-delay embeddings allow for some flexibility in building a ROM depending upon 
the scenario. Time-delay embeddings also allow one to handle latent variables when 
the full state measurements are unknown or unavailable. For a long time-delay em- 
bedding with known or unknown governing equations, one can augment the DMD al- 
gorithm by producing a time-delay coordinate system which helps make the dynamics 
linearly dominant (HAVOK-Koopman). A short time delay can be used to determine 
the rank of the underlying dynamics and potentially build a SINDy model (HAVOK- 
SINDy). Alternatively, a long time-delay embedding can discover the intrinsic rank 
and linearize the dynamics in the time-delay coordinates. For more details on DMD, 
its variants, and its broad applications, please see [78]. For a broader overview of data- 
driven methods and machine learning applied to dynamics, please see [24]. 

The diversity of strategies is important in modern complex systems simulations 
where often the equations are only partially known, but where rich measurement 
data may be available. Thus data-driven strategies can bridge the gap between mea- 
surement space and model space. Table 7.1 gives a summary of the various current 
techniques. It is envisioned that refinement and innovations using the various strate- 
gies will greatly aid in modeling the challenge problems in many fields where high- 
dimensional, multiscale physics are prevalent. Figure 7.7 gives a summary of the 
decision space necessary when considering an appropriate ROM. One can either em- 


(a) a3 (b) 

F y(a) 
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ai a2 ar 


Linear subspace (POD) Nonlinear manifold 


Figure 7.7: Low-order modeling of fluid flows begins with an appropriate coordinate system that 
captures the few dominant flow mechanisms that are dynamically relevant. It is most common to 
embed high-dimensional fluid data in a linear subspace, for example using POD (a). However, for 
the flow past a cylinder, it is clear that the data live on a low-dimensional manifold in the embedding 
space (b). Both approaches have been explored extensively, for example by Noack et al. [98] and 
Loiseau et al. [82]. After an appropriate coordinate system is obtained, there are several choices for 
model construction. 
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bed in a linear space or in a nonlinear space (manifold), and then determine the 
appropriate nonlinear dynamics. This can be done in a variety of ways depending on 


wether the underlying governing equations are known, or if only measurement data 
are available. 
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